shichao-an/notes

Shichao's Notes

notes

Documentation and source code for Shichao's Notes.

notes's Issues

some comments on block/sector size ( ch 14 )

to https://github.com/shichao-an/notes/blob/master/docs/lkd/ch14.md :
I'm not sure whether that fits as book comments, but since it was probably the best introduction in the matter that I could quickly find online, I want to share my two cents, that one may or may not find useful:

---

Suppose we need to find out what's the optimal buffer size for reading from a device. Or have special needs -- e.g. want to tune a ZFS filesystem recordsize for a custom on-disk database -- may be even with enabled deduplication, what will make our needs really special.

So we wonder how the system reads and writes data from/to the device. These i/o buffers would be directly related to sector or block sizes.

---

As for device block sizes -- they, of course, first come from the device itself ; look at

/sys/block/sda/queue :

    hw_sector_size
    logical_block_size
    physical_block_size

That would be the same as smartctl or hdparm would tell us:

smartctl -a /dev/sda    # grep -i block
hdparm -I /dev/sda     # grep -i block

"Physical" and "logical" here would differ e.g. for Advanced Format disks ( and we'll be more interested in physical_block_size, if it's different ).

---

Next, there's something reported by man 2 stat ( struct stat -> st_blksize ) :

stat /etc/passwd ... -> ... IO Block: 4096

This essentially comes from the filesystem. ( In short, fs/stat.c shows that st_blksize comes from the inode, the inode gets it from the superblock, and the superblock normally has it as a power of two capped at PAGE_SIZE, i.e. the memory page size. )

So basically this boils down to a memory page size. Good to know.

---

PS. This is for example how cat determines the optimal buffer size :

• https://unix.stackexchange.com/questions/17295/when-is-dd-suitable-for-copying-data-or-when-are-read-and-write-partial
• https://unix.stackexchange.com/questions/245499/how-does-cat-know-the-optimum-block-size-to-use

-- and it does not perform that bad at all:

• https://unix.stackexchange.com/questions/9432/is-there-a-way-to-determine-the-optimal-value-for-the-bs-parameter-to-dd/9492#9492

Dude, how did you manage to write these materials?

Are they collected from the Internet or created by yourself originally?

The Linux Kernel Development - 3rd edition is inaccessible.

And Understanding the Linux Kernel - third edition only contains 3 chapters. Could you fix those up or are there only those content available? Thank you so much for your collections.

Decrease of nready on (partly) wrong condition (ymmv!)

In your last version of the TCP Echo Server in chapter 6, you decrease nready whenever a new connection is available or data have been read from a connected socket, i.e. when POLLRDNORM resp. (POLLRDNORM | POLLERR) is set.
Now according to the manpage of poll, the return value of poll (nready) is the number of pollfds, which have a non-zero revents.
Your example might be semantically equivalent, I did not check the exact conditions. But for a cleaner version I'd recommend decreasing nready whenever (client[x].revents != 0).

What could happen otherwise is that nready is decreased not often enough, thus we check more descriptors which we have to -- which, admittedly, does not break anything. Thus, you could also argue that decreasing nready only on the flags you really want to test is faster and serves as a good heuristic.
YMMV, but it's maybe something to think about.

Copyright Infringement?

docs/spec contains material that looks verbatim from my book Systems Performance: Enterprise and the Cloud. I understand it is possible to quote excerpts for use, and there is a threshold beyond which it is copyright infringement. I can ask my publisher to investigate, but I'll ask you first: what your thoughts are on this?

Contact request, is all this is

Hi Shichao,

I'd just like to get in touch with you, wasn't sure how else to do so!?
This is reg two things:
a) i am the author of 'Hands-On System Programming with Linux', Packt, Oct 2018; was wondering if you;re interested in reading and putting it on your booklist
b) am writing a second book on kernel development & would like to speak with you reg your helping as a technical reviewer..
Pl give me an email addr where i can contact you..

Thanks & Regards,
Kaiwan
(pl delete this issue once we're in contact :))

Can others contribute too?

License Terms?

Well done Shichao!

I'm very impressed by the quality of the notes: short and concise text that managed to build up each subject from the ground up.

I was interested in reusing the notes on x86 assembly in a documentation/examples project I'm curating (the project reuses many third party code and text, with due mentioning of authors, source material and license terms). But I can't see a license attached anywhere here.

Since my project is hosted on GitHub, I'd need to include/mention the license terms of any third party resources I include or somehow reuse. Also, I'd need to know if I could apply changes to the original text, and create derivative works from it (eg: splitting it up into sub-documents, updating the text, add images, etc.).

LKD notes are not accessible anymore

LKD collection is awesome but couldn't find those notes anymore. What could be the issue?

Incorrect goroutine of execution in Chapter 8 session Example: Concurrent Echo Server

In this session at:
https://notes.shichao.io/gopl/ch8/#example-concurrent-echo-server

We have the example of reverb:

func handleConn(c net.Conn) {
    input := bufio.NewScanner(c)
    for input.Scan() {
        echo(c, input.Text(), 1*time.Second)
    }
    // NOTE: ignoring potential errors from input.Err()
    c.Close()
}

This snippet will have blocking echo, and the text explains:

The arguments to the function started by go are evaluated when the go statement itself is executed; thus input.Text() is evaluated in the main goroutine.

But in fact, the handleConn is executed in a separated goroutine that's spawned from main goroutine. So it should be clear that input.Text() is executed from that goroutine rather than the main one.

Please correct me if I'm wrong. I'm new to Go and reading this part causes me confused.
Thanks.

awesome work!

just wanted to say thanks!

is the note taking still going on?

Continuation of backlog

Any plan for continuing this repo ?

How to generate pdf from these notes?

Have you ever considered to generate the pdf version about these notes?

What tools did you use to make this website?

This is an awesome work Shichao! I am thinking of doing the same thing.

Was wondering how did you create such website? Is it custom made or you use stuff likes Jekyll or Hugo or something?

Thank you in advance!

Is there any plan to update linux kernel note

hi,I have read your blog about Linux kernel,it's awesome。but it seems it's not finished yet,(for example ,the note about VFS).do you have any plan to finish this? please let me know,thanks again for your awesome stuff。