diff options
author | Aaron Ball <nullspoon@iohq.net> | 2015-05-31 01:07:36 -0600 |
---|---|---|
committer | Aaron Ball <nullspoon@iohq.net> | 2015-05-31 01:07:36 -0600 |
commit | 3f19c2313feb53e3d7c861e317d1d75552c6cb89 (patch) | |
tree | 2750ac2865333ea03849a947776628846309d35d /src | |
download | oper.io-3f19c2313feb53e3d7c861e317d1d75552c6cb89.tar.gz oper.io-3f19c2313feb53e3d7c861e317d1d75552c6cb89.tar.xz |
Initial commit of some posts
First iterration of buildpage is written. It works okay.
Commiting header and footer html docs as well as style.css (fairly functional
port from the old site)
Also including four asciidoc syntax versions of the old wiki content.
Diffstat (limited to 'src')
-rw-r--r-- | src/Aol_Email_Hacked.ascii | 237 | ||||
-rw-r--r-- | src/Cool,_Fun,_and_Mostly_Useless_Things_to_do_with_Linux.ascii | 134 | ||||
-rw-r--r-- | src/Puppet:Out_of_Range_for_Type_Integer.ascii | 116 | ||||
-rw-r--r-- | src/Xkcd:1110.ascii | 67 |
4 files changed, 554 insertions, 0 deletions
diff --git a/src/Aol_Email_Hacked.ascii b/src/Aol_Email_Hacked.ascii new file mode 100644 index 0000000..a5121a3 --- /dev/null +++ b/src/Aol_Email_Hacked.ascii @@ -0,0 +1,237 @@ +Aol Email Hacked +================== +:author: Aaron Ball +:email: nullspoon@iohq.net + +== {doctitle} + +About four days ago, my dad's email account began spamming me. I initially +thought the email looked fishy, but it had a few things about it that made it +seem relatively legitemate. The first reason being that my dad frequently sends +me news articles (the email had a link to a "news" article, albeit a suspicious +one). The second was that the people included on the email were all people he +knows. And thus, I clicked the link on my phone and it promptly took me to a +website that downloaded the file "security.update.apk" to my phone. I said to +myself, "Self, that looks like something bad. Better not install that." + +And so I didn't. After seeing the malicious file download though, I went back +to my "dad's" email and had a good look at the headers and there it was: +several non-Aol mail servers in line, ending with my server which didn't mark +it as spam for a very key reason. + + +== The Problem + +Most people don't know that the to, cc, bcc, subject, and body are not the only +fields you can change in an email. Many who run their own mail servers for the +first time have an epiphany that they can change any field on an email, +including the *from* field. So what's to keep us from framing Roger Rabbit? +It's very easy to send an email from someone else without actually being logged +in to their account. The server conversation for that scenario would go roughly +like this... + +[quote] +____ +**super_sketchymail.info**: Pssst. Hey you over there...I have a letter for +you. _*sulks into the shadows*_ + +**gmail.com**: Okay? Lemme see... Oh look. It's a letter to +frank15689@gmail.com and it's from james19875@aol.com. Okay! I'll go deliver +this to him. + +**super_sketchymail.info**: _*continues handing out false letters*_ +____ + +There might be a subtle something you missed in that conversation just now. The +email is coming from __super_sketchymail.info__, but the letter itself says +it's from aol.com. The point here is that Gmail missed that it was a fraudulent +email and now Frank has it in his inbox. + + +== The Solution: SPF + +There are many methods to detect and respond to fraudulent emails. One of them +(the topic of this post) is this great thing invented by the elders of the +internet called SPF, or **s**ender **p**olicy **f**ramework. In a scenario +where SPF was implemented, the mail server conversation would go roughly like +this... + +[quote] +____ +**super_sketchymail.info**: Pssst. Hey you over there...I have a letter for +you. _*sulks into the shadows*_ + +**gmail.com**: Okay? Lemme see... Oh look. It's a letter to +frank15689@gmail.com and it's from james19875@aol.com. Lemme check with aol.com +first to make sure they say _super_sketchymail.info_ can send email on their +behalf + +**gmail.com**: Hey **aol.com**, can *super_sketchymail.info* send email on your +behalf? + +**AOL.com**: No they cannot! + +**gmail.com**: Nope! They say you can't. Sorry pal, I'm not going to +deliver this. +____ + +Effectively what SPF provides is a way for a mail server to verify that the +server delivering the mail is approved to do so for the given email address +(the _from_ field). In the previous conversation, super_sketchymail.info was +trying to deliver mail on behalf of Aol. Gmail then checked with Aol (their +SPF records) and saw that their list of approved mail servers did not include +super_sketchymail.info, and thus the email would not be delivered. + +Isn't that a great little bit of functionality? + + +[[where-aol-went-wrong]] +== Where AOL Went Wrong + +[[the-technical-version]] +=== The Technical Version + +The functionality I just described is really great...if you have it in +place. Aol _does_ have it in place, just not correctly. A quick lookup +of their DNS and we'll see why. + +**Note** that this DNS lookup is as of 2014.04.21. + +---- +$ dig -t txt aol.com + +; <<>> DiG 9.9.2-P2 <<>> -t txt aol.com +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32129 +;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 + +;; OPT PSEUDOSECTION: +; EDNS: version: 0, flags:; udp: 4000 +;; QUESTION SECTION: +;aol.com. IN TXT + +;; ANSWER SECTION: +aol.com. 3600 IN TXT "v=spf1 ptr:mx.aol.com ?all" +aol.com. 3600 IN TXT "spf2.0/pra ptr:mx.aol.com ?all" + +;; Query time: 62 msec +;; SERVER: 172.20.0.40#53(172.20.0.40) +;; WHEN: Wed Apr 23 08:39:02 2014 +;; MSG SIZE rcvd: 228 +---- + + +The key bits in there are the two lines that have "spf1" and spf2.0" and +end with "?all". Thos two DNS entries say a bit more than we'll discuss +here, so the most important bit in there for the purposes of this post +is the **?all**. What that says is that any host who doesn't match any +of the previous policies in any way, mark as neutral. When a server +checks Aol's DNS entries to confirm if a server is approved to send +emails, instead of saying an emphatic __no__, it says "__Yeah, sure. +Whatever__". I think that flag could be better described as the +ambivolent flag. + +The bit that ends an spf record (the _all_ bit) can have one of four +qualifiers: +, ?, \~, and -. Most SPF records (arguably all) should end +with _-all_ because that disowns all mail servers that don't match the +previous policies. Aol uses the __?all__, which is neutral (as +mentioned). + + +[[the-less-technical-version]] +=== The Less Technical Version + +Basically, the way AOL has their DNS SPF records configured, they almost +approve anyone to send mail as Aol. I say _almost approve_ because they take +only a neutral standpoint on any server that tries to send mail as them. This +is a huge problem because anyone who runs a mail server can spoof headers, send +mail as Aol users, and services like Gmail can't verify that it's not from Aol, +because Aol says that it's okay for any server to send mail as them. + +The quick solution here for Aol is to flip that *?all* to a **-all**. My guess +is that Aol has some vendors sending mail as them and they haven't taken the +time to put their vendors servers in DNS (easily fixable with the INCLUDE +mechanism). Other than that though, there's really no reason to have the ?all +in place that I can think of (besides just not knowing how spf works). + + +[[one-final-issue]] +== One Final Issue + +Despite Aol's DNS mis-configuration, there is one final issue that I can't +really speak much to. It goes back to the emails I've been receiving from my +"dad's" email account. Each of those is written to people from his contact +list, which indicates that someone was able to get in to Aol (or their user +data got out) and acquire user's contact lists. If they got their contact lists +though, who knows what else they were able to get. + +How big was this breach? I can't say. Aol +http://techcrunch.com/2014/04/21/aol-mail-hacked-with-spoofed-accounts-sending-spam/[confirmed +the breach] just two days ago. Hopefully Aol doesn't play this out poorly and +try to keep everyone in the dark. I'll post back here as I learn more. + + +[[update-2014.05.11]] +== Update: 2014.05.11 + +It's actually been a while since the issue was "resolved", I just haven't had a +chance yet to post back on it. Now though, it's snowing outside (in spring), I +have a hot mug of coffee, and my cat is sleeping on the recliner instead of my +keyboard. Let's get started. First, let's have a look at AOL's DNS to see how +they've done fixing it up. + + +---- +$ dig -t txt aol.com + +... +;; ANSWER SECTION: +aol.com. 1942 IN TXT "v=spf1 ptr:mx.aol.com include:spf.constantcontact.com include:aspmx.sailthru.com include:zendesk.com ~all" +aol.com. 1942 IN TXT "spf2.0/pra ptr:mx.aol.com include:spf.constantcontact.com include:aspmx.sailthru.com include:zendesk.com ~all" +... +---- + + +It looks like they've certainly thoroughly updated their DNS. In application, +their fix _should_ prevent folks from being able to spoof legitemate AOL +accounts, but that's actually only because of their vendors having their DNS +configured properly. To be extra clear, the reason the problem is fixed is not +because AOL has actually implemented a solid fix. As mentioned earlier in +link:#The_Technical_Version[ the technical version section], there are four +qualifiers for the trailing _all_ bit, AOL chose to use the **~**, a soft fail. +This will still not disown non-AOL server sending mail as AOL. It will only +"raise suspicion" for those emails. However, thanks to their vendors knowing +what they're doing (aspmx.sailthru.com and at least), their spf records +actually end with a __-all__, or a hard fail. + +To give a simple overview of how AOL's DNS works now, they basically include +all of their vendor's spf records in their own spf record. That means that if +any of their vendors break their own DNS to allow anyone to spoof the vendor, +the "spoofers" can also apoof AOL users because AOL's DNS is including the +vendor's bad DNS configuration. In this case though, one of AOL's vendors +(aspmx.sailthru.com), ends with a __'-all__, causing AOL's DNS configuration to +be secure becuase one of their vendors made an alright decision in their +configuration. Dear AOL... + +One final thing to note regarding the remainder of the breach. +http://www.pcworld.com/article/2148523/aol-traces-mystery-spam-to-security-breach.html[AOL +has confirmed] that there was indeed a security breach wherein the attackers +gained access to user's complete address books (email address, names, physical +mailing addresses, etc) as well as encrypted security questions/answers and +encrypted passwords (gosh I hope they mean hashed instead of encrypted +passwords). I hope that AOL comes out with a detailed report as to how the +attackers gained access to their systems. Given their mishap with DNS (benefit +of the doubt), I hope the hack on their servers wasn't nearly as obvious. Also +I'd like to know for my own edification. Due to this leak, I have begun +receiving an increased amount of non-AOL spam as if my email address was +released to more spammers. Thanks AOL. I guess though that it was bound to +happen sometime by someone. Why not AOL.. At least I got to learn +link:Exim_Spam_Filtering_with_Bogofilter[how to set up a spam filter for Exim]. + +Category:Aol +Category:Security +Category:DNS + + +// vim: set syntax=asciidoc: diff --git a/src/Cool,_Fun,_and_Mostly_Useless_Things_to_do_with_Linux.ascii b/src/Cool,_Fun,_and_Mostly_Useless_Things_to_do_with_Linux.ascii new file mode 100644 index 0000000..053239e --- /dev/null +++ b/src/Cool,_Fun,_and_Mostly_Useless_Things_to_do_with_Linux.ascii @@ -0,0 +1,134 @@ +Cool, Fun, and Mostly Useless Things to do with Linux +===================================================== +:author: Aaron Ball +:email: nullspoon@iohq.net + + +== {doctitle} + +I frequently find myself on a weekend without much to do, wishing I could have +some cool new project to put on my Linux server at home. I've implemented +several of my ideas and occasionally come up with one more, but when I'm +starving for one and don't have one on the cooker at the moment, finding a cool +Linux project idea on the internet can be quite the task so much so that you +can make searching for one a weekend project in itself...until of course you +get so frustrated with not finding one that you want to commit some uncivilized +acts to various (hopefully) inanimate objects near you. + +If that describes you, hopefully this post will help. + + +[[linux-general]] +=== Linux General + +Not all of these projects require a "Linux server" or a "Linux desktop". +Most of these are quite doable with either category. + +* Synchronize your Android phone with your Linux, Windows, Mac, or Unix systems + using https://syncthing.net/[Syncthing], the peer to peer, distributed, + self-hosted synchronization client/server. + +* Write a script in your language of choice that uses + https://rsync.samba.org/[rsync] and ssh as the transfer protocol/client, to + backup each of your linux or unix boxes (rsync is available for both). + + +[[linux-as-a-server]] +=== Linux as a Server + +This category we can safely say requires a server with a static ip +address (or at least dynamic dns), and will be running most of the time, +hence the server category. These would be good on an internal network +where you control the gateway and/or dhcp, or on an external internet +with a vps or business-class internet (most ISPs don't give out static +IPs unless they are explicitely requested). + +* Build an IRC server using http://www.ircd-hybrid.org/[ircd-hybrid] + +* Build an XMPP/jabber server for chatting with your GMail friends (or + whoever else uses xmpp) using http://www.ejabberd.im/[ejabberd] or + http://jabberd2.org/[Jabberd2] (this is the one I use) + +* Build a Ventrilo server using http://www.ventrilo.com/[Ventrilo] (duh) + +* Take webcam photos from command line using the + http://www.firestorm.cx/fswebcam/[fswebcam] package + +* Set up a motion-detecting webcam using the + http://www.lavrsen.dk/foswiki/bin/view/Motion/WebHome[motion] package + +* Build a media server that works with your PlayStation 3 using + http://mediatomb.cc/[MediaTomb] + +* Set up an IRC bot to connect to previously made IRC server using + http://www.eggheads.org/[eggdrop] + +* Build a DLNA media streaming server using + https://wiki.archlinux.org/index.php/MiniDLNA[miniDLNA] (recently renamed + http://sourceforge.net/projects/minidlna/[ReadyMedia]) and use it to stream + your music, video, and pictures to your Playstation 3, Windows computer, + Android phone, or whatever you want that supports UPnP. + + +[[linux-as-a-desktop]] +=== Linux as a Desktop +* Download all of your email locally using http://offlineimap.org/[offlineimap] + and set up command line mail using http://www.mutt.org/[mutt]. + +* Encrypt some or all of your home directory using http://ecryptfs.org/[eCryptfs] and symlinks. + +* Learn a new window manager, preferably something complicated and fun like + http://i3wm.org/[i3] (it's a tiling window manager that's designed for + keyboards). + +* Learn to use http://tmux.sourceforge.net/[tmux] (terminal multiplexer) like a + pro (attach, detach, new panes, new windows, split windows, etc). + +* Get a usb stick or external hard drive and install Linux on it, + bootloader and all. + http://en.wikipedia.org/wiki/Universal_Serial_Bus#USB_2.0_.28High_Speed.29[USB + 2.0] transfers at roughly 35 MB per second and + http://en.wikipedia.org/wiki/Universal_Serial_Bus#USB_3.0_.28Super_Speed.29[USB + 3.0] transfers at roughly 400 MB per second, so you won't see too bad of + performance. + +** I'd like to take a second to expound on this one because it's been + particularly useful to me. Having this kind of a setup allows me to use any + computer hardware that supports booting from USB as my computer. It's great + for fixing friend's computers, it's great in case your laptop battery dies, + it's more portable than a laptop (assuming you'll have a computer you can + use whever you're going), you can run Linux at work without angering the + desktop team who built your work computer. When you go on trips, you don't + have to bring both your personal laptop and your work laptop. You just need + one computer and your little hard drive. It's really a handy thing to have. + +* If your work uses Cisco VPN, you can go in using + http://www.unix-ag.uni-kl.de/~massar/vpnc/[vpnc] (Gnome and KDE GUI + implementations are readily available). + +** **Disclaimer**: I am not responsible for any repercussions of doing this. + Think before doing this one. Some companies have policies against computers + they didn't build being on their networks. + +* Write http://docs.ansible.com/index.html[Ansible] scripts to build any number of these projects. Another good + option is to write ansible scripts to build out any of your Linux laptops + (sotware manifests, adding users with consistend uids/gids, sudoers configs, + etc). I'm not trying to start a flame war. Ansible is just the easiest to get + set up without needing a server already running (okay fine, there's always + puppet apply and chef solo). + +* Learn to install https://www.archlinux.org/[Arch Linux]. That may sound + trivial, but if you haven't been doing Linux for long, or are familiar with + the standard desktop distro installation process, this can be very a very + informative project. Suffice to say, afterwards you will understand much more + about how your computer and Linux operate. The beginners' guide can be found + https://wiki.archlinux.org/index.php/Beginners%27_guide[here]. + + +Hopefully that's enough to get you started. Enjoy! + + +Category:Linux + + +// vim: set syntax=asciidoc: diff --git a/src/Puppet:Out_of_Range_for_Type_Integer.ascii b/src/Puppet:Out_of_Range_for_Type_Integer.ascii new file mode 100644 index 0000000..e92b002 --- /dev/null +++ b/src/Puppet:Out_of_Range_for_Type_Integer.ascii @@ -0,0 +1,116 @@ +Puppet:Out of Range for Type Integer +==================================== +:author: Aaron Ball +:email: nullspoon@iohq.net + +== {doctitle} + +This week we ran into a rather small (or is it large) problem with our puppet +instance. We logged into the puppet console and noticed that there were over +37,000 pending tasks, and the list was growing fast. Checking the logs, we saw +an "out of range" exception. An out of range exception for an enterprise +product is never a good thing. It's almost as bad as a segmentation fault in an +enterprise product, something you can do nothing about if you don't have access +to the source code. In this case though, we actually can do something about +this particular issue. + +Here's the exact error we were seeing... + +---- +2015-04-17T22:30:15+0000: [Worker(delayed_job.7 host:http://foosite.com pid:17446)] Class#create_from_yaml failed with ActiveRecord::StatementInvalid: PG::Error: ERROR: value "2147716789" is out of range for type integer: INSERT INTO "resource_events" ("audited", "desired_value", "historical_value", "message", "name", "previous_value", "property", "resource_status_id", "status", "time") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) RETURNING "id" - 2 failed attempts 2015-04-17T22:30:15+0000: [Worker(delayed_job.7 host:http://foosite.com pid:17446)] PERMANENTLY removing Class#create_from_yaml because of 3 consecutive failures. +2015-04-17T22:30:15+0000: [Worker(delayed_job.7 host:http://foosite.com pid:17446)] 1 jobs processed at 0.3037 j/s, 1 failed ... +2015-04-17T22:30:15+0000: [Worker(delayed_job.2 host:http://foosite.com pid:17361)] Class#create_from_yaml failed with +ActiveRecord::StatementInvalid: PG::Error: ERROR: value "2147716814" is out of range for type integer +---- + + +[[solution]] +=== Solution + +It turns out that the functionality that uses this is deprecated as of early +2014, so this supposedly isn't an issue with newer puppet installs. However, if +you're using an older puppet (3.0 or older), you might run into this problem. + +The problem lies in the database schema for the puppet console. Basically, +every time a node checks in, it inserts a row into the database. The database +has some tables with columns that auto-increment (0, 1, 2, 3, etc). If you have +a lot of nodes reporting back frequently, this number will likely increase a +lot over time. In our case, we have 333 nodes reporting every 30 minutes or +more (we do development and thus we often manually run puppet agent with the -t +switch). In our case, to hit 37,000, it would have taken a little over 2 days +(30*(24*60)*333 = 1 day's checkin count) + +The columns that autoincrement use the int datatype. This datatype, as seen +http://www.postgresql.org/docs/9.1/static/datatype-numeric.html[here], uses 4 +bytes. In case anyone doesn't remember, there are 8 bits in a byte, which means +that +4 * 8 = 32+. That means that the maximum number that will fit +in any column with the int data type is +2^(32-1)^+, which equals +2,147,483,648. That means 2 billion puppet reports. It seems like a number not +easy to achieve, but it is quite possible - we did it. + +The solution here is to change the data type on the columns in concern to be +bigint rather than integer. Again, as documented by the postgres folks +http://www.postgresql.org/docs/9.1/static/datatype-numeric.html[here], a bigint +is 8 bytes, which is a 64 bit number. That means the largest it can hold is +9,223,372,036,854,775,807 (about 9 quintillion). That said, let's get to it. + + +[[executing-the-fix]] +== Executing the Fix + +Before performing the fix, we should probably perform a backup of the database, +unless you're the type who likes causing a fire you have to put out on a +Friday, just like... + +image:files/Most_interesting_table_schemas.jpg[Compliments of Gimp,title="Compliments +of Gimp"] + +To execute a backup (we'll assume your database name is **console**), run + +---- +pgsql_dump -U console -W -h localhost console > console.$(date '+%Y%d%m.%H%M').sql +---- + +Once that backup is complete (mine was 86 GB, so it took a while), shut down +all of your puppet services to be safe. A list of services you might want to +shut down can be found +https://docs.puppetlabs.com/pe/latest/install_what_and_where.html#services[here]. +A general rule of thumb though is, anything in /etc/init.d/ that starts with +_pe-_ is something that should be stopped, **excepting pe-postgresql**. + +Once that's done, execute this fun console trick. + +---- +$ psql -U console -W -h localhot + +-- Change to the console table console=> \c console + +-- This one might take a *very* long time (mine took an hour) console=> +alter table resource_statuses alter column id type bigint; console=> +alter table resource_events alter column id type bigint; console=> alter +table resource_events alter column resource_status_id type bigint +console=> \q +---- + +With that, restart the _pe-postgresql_ service for good measure. Once that's +done restarting, start up the other pe-* services and everything should be +working now. + + +[[related-documentation]] +== Related Documentation + +This is a bug that was reported about three years ago. They have since migrated +ticket tracking systems, so the links can be found at... + +* (old ticket system) https://projects.puppetlabs.com/issues/9225 +* (new ticket system) https://tickets.puppetlabs.com/browse/PUP-1173 + + +Category:Linux +Category:Automation +Category:Puppet +Category:Postgres + + +// vim: set syntax=asciidoc: diff --git a/src/Xkcd:1110.ascii b/src/Xkcd:1110.ascii new file mode 100644 index 0000000..83f8d9a --- /dev/null +++ b/src/Xkcd:1110.ascii @@ -0,0 +1,67 @@ +Xkcd:1110 +========= +:author: Aaron Ball +:email: nullspoon@iohq.net + +== {doctitle} + +I really like the webcomic http://xkcd.com[xkcd]. Its author, Randall, is +hilarious. If you don't read this comic, you definitely should. + +Recently Randall http://xkcd.com/1110[drew one] that blew my mind (seriously, +there are brains everywhere). He basically made what looks to be a 100x100 +(there are some empty tiles in there so that's not super accurate) grid of a +sad, yet wonderful world. This world, populated by javascript, will take you a +tremendous amount of time to scroll through. I can only imagine how much time +this took him to make. + +Well, not to put all of that work to waste, but I decided I wanted to assemble +the entire grid into a single image. The first step to that is to download the +entire grid of images. With that, I wrote a script. + +Currently, that script is downloading all of that commic with a .2 second sleep +time between images (no DOSing for me). I will post back here with a zip file +containing every image and as soon as I have the time, I will write a script to +automagically assemble the entire thing! I will also post that here. + +However, first things first (as I said). The first script to download the +entire commic looks like so (yes, I'm sure there are more efficient ways to do +this) + +---- +#!/bin/bash +for n in {0..50..1}; do + # Quadrant 1 + for e in {0..50..1}; do + wget "http://imgs.xkcd.com/clickdrag/"$n"n"$e"e.png" && echo $n"n"$e"e.png" + sleep .2; + done + + # Quadrant 2 + for w in {0..50..1}; do + wget "http://imgs.xkcd.com/clickdrag/"$n"n"$w"w.png" && echo $n"n"$w"w.png" + sleep .2; + done +done + +for s in {1..50..1}; do + # Quadrant 3 + for w in {0..50..1}; do + wget "http://imgs.xkcd.com/clickdrag/"$s"s"$w"w.png" && echo $s"s"$w"w.png" + sleep .2; + done + + # Quadrant 4 + for e in {0..50..1}; do + wget "http://imgs.xkcd.com/clickdrag/"$s"s"$e"e.png" echo $s"s"$e"e.png" + sleep .2; + done +done +---- + +Category:xkcd +Category:Linux +Category:Scripting + + +// vim: set syntax=asciidoc: |