Creating instances is slow

[shakenfist]

Creating instances is slow

[mikalstill]

This doesn't seem true any more, almost all of the boot time is IO (fetching image, resizing image):

$ sf-client instance events 3b2a4480-67ef-4ff8-a2a6-ab8817451fce
+----------------------------+------+----------------------------+-----------------------------+---------------------+--------------------------+
|         timestamp          | node |         operation          |            phase            |       duration      |         message          |
+----------------------------+------+----------------------------+-----------------------------+---------------------+--------------------------+
| 2020-07-15 09:31:44.371041 | sf-1 |       uuid allocated       |             None            |         None        |           None           |
| 2020-07-15 09:31:44.449057 | sf-1 |          schedule          |            start            |         None        |           None           |
| 2020-07-15 09:31:44.455636 | sf-1 |          schedule          |      Initial candidates     |         None        | ['sf-1', 'sf-2', 'sf-3'] |
| 2020-07-15 09:31:44.461789 | sf-1 |          schedule          |    Have enough actual CPU   |         None        | ['sf-1', 'sf-2', 'sf-3'] |
| 2020-07-15 09:31:44.468023 | sf-1 |          schedule          |     Have enough idle CPU    |         None        | ['sf-1', 'sf-2', 'sf-3'] |
| 2020-07-15 09:31:44.473598 | sf-1 |          schedule          |     Have enough idle RAM    |         None        | ['sf-1', 'sf-2', 'sf-3'] |
| 2020-07-15 09:31:44.480172 | sf-1 |          schedule          |    Have enough idle disk    |         None        | ['sf-1', 'sf-2', 'sf-3'] |
| 2020-07-15 09:31:44.519683 | sf-1 |          schedule          | Have most matching networks |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.531199 | sf-1 |          schedule          |  Have most matching images  |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.537601 | sf-1 |          schedule          |            finish           | 0.08812952041625977 |           None           |
| 2020-07-15 09:31:44.555803 | sf-1 |         placement          |             None            |         None        |           sf-3           |
| 2020-07-15 09:31:44.609718 | sf-3 |          schedule          |            start            |         None        |           None           |
| 2020-07-15 09:31:44.637330 | sf-3 |          schedule          |      Forced candidates      |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.643363 | sf-3 |          schedule          |      Initial candidates     |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.648914 | sf-3 |          schedule          |    Have enough actual CPU   |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.654865 | sf-3 |          schedule          |     Have enough idle CPU    |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.660918 | sf-3 |          schedule          |     Have enough idle RAM    |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.665908 | sf-3 |          schedule          |    Have enough idle disk    |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.687299 | sf-3 |          schedule          | Have most matching networks |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.697811 | sf-3 |          schedule          |  Have most matching images  |         None        |         ['sf-3']         |
| 2020-07-15 09:31:44.702994 | sf-3 |          schedule          |            finish           | 0.09373188018798828 |           None           |
| 2020-07-15 09:31:45.404208 | sf-3 |   ensure networks exist    |            start            |         None        |           None           |
| 2020-07-15 09:31:45.636326 | sf-3 |   ensure networks exist    |            finish           | 0.23077821731567383 |           None           |
| 2020-07-15 09:31:45.642094 | sf-3 |     instance creation      |            start            |         None        |           None           |
| 2020-07-15 09:31:45.647934 | sf-3 |     make config drive      |            start            |         None        |           None           |
| 2020-07-15 09:31:45.687532 | sf-3 |     make config drive      |            finish           | 0.03952956199645996 |           None           |
| 2020-07-15 09:31:45.691971 | sf-3 |        fetch image         |            start            |         None        |           None           |
| 2020-07-15 09:34:05.453902 | sf-3 |        fetch image         |            finish           |  139.54655241966248 |           None           |
| 2020-07-15 09:34:05.633676 | sf-3 |      transcode image       |            start            |         None        |           None           |
| 2020-07-15 09:34:05.665518 | sf-3 |      transcode image       |            finish           | 0.12225174903869629 |           None           |
| 2020-07-15 09:34:05.670520 | sf-3 |        resize image        |            start            |         None        |           None           |
| 2020-07-15 09:34:26.338176 | sf-3 |        resize image        |            finish           |  20.662424087524414 |           None           |
| 2020-07-15 09:34:26.344269 | sf-3 | create copy on write layer |            start            |         None        |           None           |
| 2020-07-15 09:34:26.381421 | sf-3 | create copy on write layer |            finish           |  0.0349123477935791 |           None           |
| 2020-07-15 09:34:26.397727 | sf-3 |     create domain XML      |            start            |         None        |           None           |
| 2020-07-15 09:34:26.438484 | sf-3 |     create domain XML      |            finish           |  0.0406184196472168 |           None           |
| 2020-07-15 09:34:26.443619 | sf-3 |       create domain        |            start            |         None        |           None           |
| 2020-07-15 09:34:27.714417 | sf-3 |       create domain        |            finish           |  1.2696380615234375 |           None           |
| 2020-07-15 09:34:27.730579 | sf-3 |     instance creation      |            finish           |  162.08872747421265 |           None           |
| 2020-07-15 09:34:27.768235 | sf-1 |            api             |        get interfaces       |         None        |           None           |
| 2020-07-15 09:37:23.368603 | sf-1 |            api             |          get events         |         None        |           None           |
+----------------------------+------+----------------------------+-----------------------------+---------------------+--------------------------+```