gh2o/digitalocean-debian-to-arch

Script to convert a Debian installation on DigitalOcean to Arch Linux

digitalocean-debian-to-arch's Issues

Archlinux changed handling of local trusted certs

on 12/11/2014 Archlinux changed the way it handles local CA certs:
https://www.archlinux.org/news/ca-certificates-update/

/usr/bin/update-ca-certificates is no longer used. `trusted extract-compat' calls /usr/lib/p11-kit/trust-extract-compat which references chmod and ln. these both reside within coreutils which doesn't appear to be included currently.

Default DNS 8.8.8.8

Why is the default DNS set to 8.8.8.8? And where is this config set? By default I don't see anything in /etc/netctl or in /etc/systemd/network so I wonder...

The sane default should be to use the gateway as DNS, and not some external g00gle service.

Way to append systemd-networkd configuration?

Hi,

Thanks for the script, it just worked and I was impressed to see that you went the extra mile to use systemd-networkd and even create a PKGINFO!

Quick question/feature request: Is there a good way to append to the /run/systemd/network/dosync-eth0.network file? I need to append Tunnel=he-ipv6 for a IPv6 tunnel service I'm using.

I hacked up digitalocean-synchronize to do this, but it doesn't feel right.

ipv6 configs

How to configurate ipv6 for Droplet with archlinux?
All configs due too wiki are missing and I even don't know how to make server been anable via ipv6

cannot type at console?

For some reason I cannot type at the console, no idea what is wrong. I can get output to appear by running the command echo "dasldsjkjdkajkls" > /dev/console. I can also read the console from the command line with cat /dev/vcs. Just cannot type anything in my browser.

Tested on multiple browsers (latest safari and chrome), I can login via ssh and use it perfectly. I tried power cycling as well. Digital ocean support said they can't help me because I used this script.

Any ideas?

Enforce IPForward=kernel easily

As on tin. Trying to set IPForward=kernel (because systemd-networkd decided to be idiots and smother the fucking sysctl setting we already have) is a bit ridiculous. Maybe check to see if sysctl has it set already and add the line, or just add IPForward=kernel outright?

Number of inodes

Thank for script! Please add an option to configure number of inodes, more precisily bytes-per-inode ratio. It is -i parameter of mkfs command.

Sync script hangs on boot

After updating a box and rebooting, the instance is not stuck on

A start job is running for DigitalOcean Synchronization

with no limit. The instance simply won't boot.

Any fix for this situation?

Fatal error right before installation

Hello,

I got these errors after the script downloaded all the packages and just before styarting the install: "error: could not determine cachedir mount point /packages"
"error: failed to commit transaction (not enough free disk space)" just right before the installation.

How may I go forward please?

Thanks

Here is some infos:

uname -a => Linux v1871 2.6.32-openvz-042stab106.4-amd64 #1 SMP Fri Mar 27 15:27:00 MSK 2015 x86_64 GNU/Linux

before errors => :: Synchronizing package databases...
core 122.3 KiB 501K/s 00:00 [###########################################################] 100%
extra 1726.7 KiB 669K/s 00:03 [###########################################################] 100%
community 2.7 MiB 733K/s 00:04 [###########################################################] 100%
resolving dependencies...
looking for conflicting packages...
warning: dependency cycle detected:
warning: ncurses will be installed before its bash dependency

Packages (128) acl-2.2.52-2 archlinux-keyring-20150523-1 attr-2.4.47-1 ca-certificates-20150402-1 ca-certificates-cacert-20140824-2 [...] vi-1:070224-1 which-2.21-1 xfsprogs-3.2.2-1

Total Download Size: 114.13 MiB
Total Installed Size: 517.09 MiB

:: Proceed with installation? [Y/n]
error: could not determine cachedir mount point /packages
error: failed to commit transaction (not enough free disk space)
Errors occurred, no packages were upgraded.
[Tue Jun 2 06:02:23 CEST 2015] Cleaning up ...

Flag: confirm "wipe this droplet"

I want to integrate this script inside an automated build system but I've to handle the confirmation after running the script:

[Mon Nov 28 19:14:59 UTC 2016] Type "wipe this droplet" to continue or anything else to cancel.

Is it possible to add a flag to the script to skip this confirmation?

busybox missing

Hi;

just in case
I used this script on a non Digital Ocean Debian 8.x

It works well except I'd need to add busybox

apt-get -y install busybox

And SSH was disabled ;)

But everything work perfectly

Startup failed

When I execute the script, the console has only the following output and cannot be started to arch.

root@debain:"# [ 565.211002) init: 32 output lines suppressed due to ratelimiting
[ 565.226695] systend-journald[4801: Failed to send strean file descriptor to service manager: Comnection refused
[ 565.2565371 systend-journald[4801: Received SIGTERM from PID 1 (bash).

Block storage?

I tried adding a block storage volume but I couldn't figure out how the volume is represented to Arch. Does using Arch mean no access to block storage volumes or is there something more that needs to be done (or, more likely, am I just being dense and missing something obvious)?

Attempting to update hostname for PTR record fails

While changing the hostname to anything else works, changing it to a domain name for a PTR record fails

The hostname is not updated on the droplet, however on digitalocean's site it is, with the PTR record showing up under Networking, however doing a reverse DNS on the IP address fails.

Official support for custom images

I don't know how much this [1] affects your project; maybe it's just a new opportunity to do things right.

[1] https://blog.digitalocean.com/custom-images/

Cheers,
Alex.

ZFS support

Hi,

It would be great to get ZFS support.

Debian 9.0 support

Debian 9.0 is now available on DO. Debian 8.x is still available but it would be great to keep things up to date :)

Unable to start droplet

Script executes without any errors but upon reconnecting PuTTy says 'conection refused'. on opening up console it shows as in screenshot. I used default branch on debian 9.x

occasionally fails to reread the partition table

every once in a while (this happens very inconsistently), the script will fail to reread the partition table after running the blockplan due to /dev/vda still being in use. this typically manifests as a hang, because the script sets -e and the blockdev call returns an error, so it never gets to the point of rebooting (and even if you reboot manually, grub won't have been set up). after the switch-root, stdout from the script is no longer going anywhere useful, so i had to track this down by adding more debug logging to stage4. not sure what the root cause is, but it's not a timing thing - retrying the blockdev call on failure will just sit in a loop retrying forever when this happens.

/etc/fstab empty

No idea if it's a problem, but fstab is left empty after the migration... maybe adding a genfstab step would be desirable?

Add instructions how to upgrade existing Debian 7 installs to Debian 8

Since version 8 doesn't have to use kexec hacks, it is probably a better idea, but having a day of downtime to migrate manually (backup & restore) is not acceptable in my case.

The administration panel supports changing the kernel, but I am not sure it would work out of the box.

Select a kernel and click change.

This will update your configuration. Then power off the server from the command line and boot it from the Control Panel and the new kernel will be active. To revert, simply select 'Original Kernel' and follow the same process.

Your droplet's current kernel: Debian 7.0 x64 vmlinuz-3.2.0-4-amd64 (3.2.65-1+deb7u1)

SSH keys can be removed after ssh key generation service starts

Not sure how to fix this, apart from adding ssh-keygen -A after we clear the keys.

Slackware install possible?

@gh2o Any chance you might do a script for installing Slackware on DO?

Feature option: create swap space

Hi,

We should be able to have the feature option --swap. For example --swap 2GB.

Thanks,
HLFH

Can't create new DO droplet from arch snapshot

Creation starts and then error msg: "Droplet create event failed, please try again".

DO support answer:
It does appear you are trying to run Arch over Debian. Once you have made the updated changes this creates interfaces that aren't compatible with cloud-init or our system. Thus trying to create a new droplet from a snapshot of this will fail. Sadly to use Arch you would have to run the install on a new droplet. Any snapshots of that droplet however won't be able to create new droplets due to the changes the script makes.

sshd not active after reboot

When I reboot my droplet, I get connection refused on port 22 when trying to connect with ssh. Only after I log in through the DigitalOcean console, does (presumably) sshd start and I can connect via ssh. Why does sshd not start before the login on the physical machine? Or should I make a dummy user with no password that automatically logs in to start this service?

No networking after reboot

First, I want to say that this script is awesome! I had to run it through the DigitalOcean console so I could get back in after it rebooted with no networking, but it's working great after I got that fixed. Not sure if the IP/gateway can be correctly detected by a script after reboot or not. I had to check the Settings -> Networking tab on the droplet page to find mine.

systemd 225 -> 226 breaks network config on all converted vm's

The system update from systemd 225 to 226 seems to have renamed network interfaces. After an update, and a reboot, all converted VMs will be offline.

I don't know if digitalocean-synchronize needs to be updated to block systemd >=226 and/or a new version that uses the "ens" interface names needs to be released, but everything is down

Security issue

wget https://raw.githubusercontent.com/gh2o/digitalocean-debian-to-arch/debian8/install.sh && bash install.sh

This isn't an acceptable way to install software unless you have severe mental disabilities.

E: typos

Lost connection to droplets

I've tried your scripts, it's work well as I expected but I found the problem, if I upgrade digitalocean-synchronize from AUR, my droplet will lost the network and I can't connect to fixing it. All things to do is rebuild the droplet from the beginning.

No network address assigned after droplet resize

I resized my arch droplet and after the reboot it is no longer reachable.
There is no IPv4 address assigned to eth0.

error setting certificate verify locations

==> Updating trust database...
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   5  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   5  signed:  62  trust: 0-, 0q, 0n, 5m, 0f, 0u
gpg: depth: 2  valid:  62  signed:   5  trust: 62-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2016-01-22
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update community (download library error)
error: failed to synchronize any databases
error: failed to init transaction (download library error)
[Wed Apr 15 01:55:46 UTC 2015] Cleaning up ...
root@a:~#                                                             

It looks like some error during repo syncing.

Any idea?

Incompatible with snapshots?

I initially setup my system with this approach about 20 days ago and it's been working wonderfully. After creating some snapshots, I've found that I'm unable to boot a new droplet. That is, I can flawlessly take down my initial droplet and spin it back up, but I'm unable to spin up any new droplets using a snapshot of this system.

I've spent a couple of days going over this with the DigitalOcean support team and they've found nothing wrong on their end so I'm lead to believe it's some bad state from this transmutation. Since my evidence for this is inconclusive, I'd really appreciate it if anyone else would try the same thing. Snap shot your "Arch" system and try to spin up a droplet based on the snapshot; if it boots and you can log into it, you get further than me.

Otherwise, having written this system, perhaps you have some more troubleshooting suggestions. I've run fsck on the droplet (from a recovery kernel) with no reported errors and nothing in lost+found.

Support for floating IPs

Floating IPs should be supported. This basically boils down to executing

ip addr add dev eth0 $(curl -sS http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address)

or crafting an equivalent configuration file for networkd. Note that floating IPs are subject to (mild) NAT—they are unshared, but the address obtained above is in 10.0.0.0/8.

Adding encrypted install functionality to the script

Solved, sorry for bothering, saw the other issue about disk encryption here.

Cannot login to machines built from a debian-to-arch image?

I cannot seem to use this to create a reusable archlinux image for myself, and I'm not sure I understand why.

When I run the script and reboot the box, I have a seemingly perfect arch droplet, so then I image it. But when I create a new droplet from the image, it refuses SSH connections (on 22) and I also cannot use the DO console because the root password has changed.

The genesis droplet works fine.

It doesn't seem to be a networking issue (the public IP responds to ping).

Any idea what I'm missing here? Thanks.

linux-lts

Great stuff! Been wanting to do this exactly. Just wondering if we can have the option to have the linux-lts installed instead of the default, that would be sweet.

Cheers.

[debian8] Cannot restore snapshot

The script of branch debian8 seems to be still incompatible with snapshots, similar to the issue stated on #7
When trying to create a new droplet from an existing arch snapshot converted from debian8, it always fails with

droplet create event failed please try again

I ticketed DigitalOcean, they said:

I see that those create attempts are running into an error with the message "Failed to configure network interfaces", which can happen if you're trying to use custom operating systems like that. It seems /etc/network/interfaces is pointing somewhere outside of the filesystem or to a different partition

But the script from branch debian8 seems to have created a doroot partition and corresponding /etc/network/interfaces in it. Is the partition probably misplaced so that DigitalOcean couldn't find this partition?

Emergency shell on boot after kernel upgrade (crc32c module not loaded)

Hi,
Today I decided to pacman -Syu my Arch system and reboot, and to my surprise I couldn't SSH and had to use the DigitalOcean console to find out it failed to load the filesystem due to missing the crc32c module, and it was dropped to an emergency shell.
The problem for me was fixed by adding crc32c to the MODULE=" ..." list in /etc/mkinitcpio.conf before running the system update.

Unfortunately, once this problem occurs It seems like your Droplet is completely hosed!
It seems there is no (good?) way to download snapshots/backups of droplets so that the boot issue can be fixed manually, so I had to restore to a snapshot I made just after running this script. While I was at it, I did test running the system upgrade from that snapshot, and the same problem existed there.

So, this problem might exist specifically on the system, the snapshot I made (22 days old, so it was made with the most up to date version of this script), or with the upgrade itself. I will test it again with a fresh Debian droplet tomorrow and run the script on it to see if it is a problem with a fresh system or not.

Luckily the Droplet that was affected for me was essentially a throwaway, hopefully this doesn't cause anyone any real grief!

Stuck for a long time after reboot

This is what the VNC console shows:

Root password after reboot?

After the installation has finished and the system rebooted, what is my root password? I'm trying to connect to the server via ssh and it won't accept the old root password from my Debian droplet (neither the default one by Digitalocean nor the one i had changed it to)

This is badass. Thank you.

That is all.

License?

Can you give this a license? Thanks.

This is badass. Thank you.

That is all. Worked like a charm.

Floating IP?

Do we have support for floating ip?

I was following this guide to setup floating ip to no avail
it seems that changes to /etc/systemd/network/static.network has no effect on /var/run/systemd/network/ens3.network?

Why GPT?

Why is GPT important? The main advantage of GPT is working with large disks (2+ TB), and digital ocean's largest primary SSD size is 640GB. Arch doesn't prefer GPT over MBR or anything either, so it seems like specifying partition table preference, and especially defaulting it to GPT introduces a lot of complexity to the script to no real benefit.

Not really a complaint or anything, I just don't understand why GPT is considered at all. I am using this script to good effect either way. Thank you for releasing/maintaining it publicly 👍 👍

Pacman error when synchronising databases

After running this on a Debian 7.0 x64 droplet, it conks out in the bootstrap_system function with this:

:: Synchronizing package databases...
error: failed retrieving file 'core.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from mirrors.kernel.org : error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
error: failed to update community (download library error)
error: failed to synchronize any databases
error: failed to init transaction (download library error)
[Tue Apr 14 09:29:26 UTC 2015] Cleaning up ...

DigitalOcean Archlinux is down

The Archlinux mirror is currently 404'ing, don't know if it'll come back.

Cryptsetup Support possible?

I saw, it is possible to use btrfs now.
So I wonder if cryptsetup with this initrd works, too.

Is the locale supposed to be C?

So, it seems to me the step of running locale-gen has been left out of the automatic installation. I assumed all was fine and inevitably ran into encoding errors (npm and its super fancy terminal blobbies). Is there any reason to suspect someone would not want en_US.UTF-8 to be the default locale?

(fantastic project regardless! ❤️ )