backups-update-internet-celebrity.md (5088B)
1 +++ 2 title = "An Update on Bad Backups and Internet Celebrity" 3 date = 2021-06-27 4 updated = 2022-06-15 5 +++ 6 7 My recent post on the [Great Alpine Migration Tragedy] of 2021 garnered 8 some serious attention! I got a number of emails, a whole host of 9 feedback on [Lobste.rs], and over **100 points** on [Hacker News]! As I 10 understand it, that means I'm now a top internet celebrity (and eligible 11 for the [10 KB Club]). Still, I must remain humble even as I catapult 12 toward digital infamy and thus I am here to respond to feedback, answer 13 questions, and talk about what I've done to prevent something like this 14 from happening again. 15 16 <!-- more --> 17 18 ## Useful Tips from Strangers 19 20 ### Ctrl-C is Not a Debugging Tool 21 22 That's a good point: I've been advised to update my methods. Several 23 folks pointed me towards [`strace`]. Sending the `SIGUSR1` or `SIGINFO` 24 signal to Tarsnap would also work. The `-v` flag could have been 25 helpful as well. 26 27 ### Don't Wipe Prematurely 28 29 A number of folks were confused as to why I didn't keep the [old 30 machine] around during the transition. Others detailed their own 31 [migration strategies][klodolph]. In general, it seems that folks agree 32 you should keep around your old machine until you're totally certain 33 everything is up and running in your new setup. 34 35 As I mention briefly in the article, I've actually done this in the past 36 myself! This time, however, there was no second server. Everything was 37 done in-place. I cannot change this, as it occurred in the past. In the 38 future I will avoid this kind of migration. 39 40 ### A Restore System 41 42 A number of people took issue with my take-aways from the experience. 43 See: 44 45 > Trying to address a lack of confidence by increasing the backup 46 > frequency doesn't make sense. The backup frequency is the most trivial 47 > thing to adjust and doesn't address deeper issues, like the fact that 48 > you need to dump/restore databases properly and shouldn't copy files 49 > from a live database. 50 > 51 > by [klodolph] 52 53 I'll start by saying that that increasing backup frequency from zero to 54 anything greater than zero does, in fact, address a significant issue. I 55 now have backups where there were none before. 56 57 Still, the point stands. Without quality backups, and quality restore 58 strategies, the backups are not as useful as they could be. 59 60 ### A Little Bit of Gold and a Pager 61 62 There was some fun memeing in the Hacker News comments in response to me 63 being a teenager. Specifically, wildmanx said that they [wouldn't hire 64 me for their "company IT or devops or whatnot."][hire]. Several people 65 have a sense of humour: 66 67 > Good idea to let him graduate from highschool before hiring him! 68 > 69 > by [nick__m] 70 71 I'll add for further memery that in addition to the stress of my entire 72 server infrastructure burning, I had to deal with passing the 73 behind-the-wheel driver's test. I've now been a licensed driver for 74 about a week and a half. Coincidentally, I got my server back running 75 the same day I passed the test. 76 77 ## How I'm Doing Backups 78 79 I moved all the data in my self-hosting setup into a single directory. 80 The files were previously all lying about my home directory, which is 81 what led to the tragic "forgetting about `.env` amongst all my dotfiles" 82 incident. 83 84 I set up a cronjob that runs the backup script [`acts`] daily, set to 85 back up the self hosting directory. It currently keeps a buffer of 31 86 daily and 12 monthly backups, and keeps around yearly backups forever. 87 It's been running for a couple days now, and it's all been smooth 88 sailing so far. 89 90 ## To Infinity and Beyond 91 92 I'd also like to say thanks to a number of people who were especially 93 kind, and made this a fun experience: 94 1. Graham Percival, an employee at Tarsnap Backup Inc., reached out to 95 me directly – we had a fun and interesting email exchange. 96 2. [hannu] was the first to write the later much-echoed sentiment that 97 reflecting on one's mistakes is a useful and laudable thing to do 98 (see also [abraae] and [argomo]). 99 100 I'm going to continue iterating my self-hosting setup. I've still got a 101 couple [new services] I might want to throw up there. I also can't 102 forget about that [dedicated page] I promised... 103 104 [Great Alpine Migration Tragedy]: @/posts/wrong-way-to-switch-server-os.md 105 [Lobste.rs]: https://lobste.rs/s/ofcfbq/wrong_way_switch_operating_systems_on 106 [Hacker News]: https://news.ycombinator.com/item?id=27562613 107 [10 KB Club]: https://10kbclub.com 108 [`strace`]: https://jvns.ca/blog/2021/04/03/what-problems-do-people-solve-with-strace/ 109 [old machine]: https://lobste.rs/s/ofcfbq/wrong_way_switch_operating_systems_on#c_sfwzdq 110 [klodolph]: https://news.ycombinator.com/item?id=27596126 111 [hire]: https://news.ycombinator.com/item?id=27597216 112 [nick__m]: https://news.ycombinator.com/item?id=27597698 113 [`acts`]: https://github.com/alexjurkiewicz/acts/ 114 [hannu]: https://lobste.rs/s/ofcfbq/wrong_way_switch_operating_systems_on#c_b0zqnt 115 [abraae]: https://news.ycombinator.com/item?id=27597436 116 [argomo]: https://news.ycombinator.com/item?id=27598224 117 [new services]: https://www.firefly-iii.org 118 [dedicated page]: @/posts/wrong-way-to-switch-server-os.md#bye-bye