by Luke Rodgers on October 13, 2013
“When David called to say he and Patty were coming for a visit, Noel never thought of saying no. And he asked me how he could compete with David. He thought David was coming to his house to win me away. After he reads more literature he’ll realize that is too easy. There will have to be complexities. The complexities will protect him forever.”
From “Vermont,” by Ann Beattie
by Luke Rodgers on August 25, 2013
I’ve been experimenting lately with using DigitalOcean both for remote dev environments (to facilitate development on a Chromebook) and also for staging/testing servers.
The plugin for Vagrant in concert with chef solo makes spinning up and provisioning new instances a breeze, but this post is about going the slightly more manual route, and using one of the application bundles (currently in beta) for Ubuntu 12.04.
These steps are a combination of things I’ve gleaned from various other places and my experience. I make no claim for their soundness.
Create the droplet
Create a new droplet using the Ubuntu 12.04 x32 (or x64–it shouldn’t matter for our purposes here), and select “LAMP on Ubuntu 12.04″ from the Applications tab.
At this point, I’m going to assume that you’ll also add your SSH public keys, so you don’t have to login with username and password credentials. (If you don’t have an SSH key, here are some instructions for generating one, you can just ignore the git-specific stuff.) An advantage of doing this is that your root password to the new droplet will not be sent over email (because there will no root password).
When you’ve selected all the options, click “Create Droplet.”
Login and Setup
Droplet creation should take a minute at most, after which you’ll see a screen with various information about your newly created vm, including an IP address. If you specified that your SSH keys should be automatically added, you should be able to SSH in now.
First off, let’s change the default mysql root password as suggested by the login banner (note that the banner may continue to say that the password is still “password” even after you’ve changed it and logged in again):
mysqladmin -u root -p'password' password newpassword
Next, because we began with our SSH keys pre-installed, there’s no root password, so set one by typing `passwd` and following the prompts.
Next we’ll issue some commands with the package manager used by Ubuntu, `apt-get` to first update the list of available packages, and then upgrade the installed ones:
apt-get update apt-get upgrade
Next, install fail2ban, a service that scans logfiles and auto-bans IP addresses that show signs of malicious activity, a good line of defense against crackers:
apt-get install fail2ban
Next, I’ll install my text editor of choice, vim:
apt-get install vim
As well as unzip…
apt-get install unzip
… and ack
apt-get install ack-grep
Adding a user
Because it is, for a variety of reasons, generally not a good idea to do things as root, let’s add a new user, create a home folder for them with the right permissions, copy the contents of root’s authorized keys files to the new user’s .ssh folder, so we can ssh in as that user, give them a password, and set their default shell to bash.
useradd luke mkdir /home/luke mkdir /home/luke/.ssh chmod 700 /home/luke/.ssh
And add the contents of the root user’s authorized_keys files to that of the new user:
cat .ssh/authorized_keys > /home/luke/.ssh/authorized_keys chmod 400 /home/luke/.ssh/authorized_keys chown -R luke:luke /home/luke passwd luke chsh -s /bin/bash luke
Now we’ll give that user the ability to run commands as root via `sudo`. Type `visudo` then enter this line, say, below the similar one for root (it doesn’t matter where, actually):
luke ALL=(ALL) ALL
Hit Command-X when done editing.
Now we’ll disable remote root login. Edit this file, /etc/ssh/sshd_config, with vim, or however you prefer, and make change “PermitRootLogin yes” to “PermitRootLogin no”, and uncomment the line “#PasswordAuthentication yes” and change it to “no”. This will mean you can only login on machines that have your SSH private key. Following that, we need to restart ssh:
service ssh restart
Now we’ll install a firewall to control which ports we allow traffic into. We’ll allow SSH and SFTP (port 22), HTTP (80), and HTTPS (443).
apt-get install ufw ufw allow 22 ufw allow 80 ufw allow 443 ufw enable
You may get a warning after the last command about this disrupting your SSH session, but you should be able to ignore it.
Other dev tools
git, rvm + ruby, tmux
apt-get install git apt-get install tmux \curl -L https://get.rvm.io | bash -s stable --ruby
These are just some relevant notes and steps I’m including mostly for myself to remember for later:
In some cases you may need FTP and not just SFTP.
apt-get install vsftpd
Edit /etc/vsftpd.conf and change the following lines
anonymous_enable=NO local_enable=YES write_enable=YES
Save the file, and `/etc/init.d/vsftpd restart` then `ufw allow 21`.
Serving a git repository
If you’re serving files directly from a git repository, make sure you aren’t serving .git.
by Luke Rodgers on August 2, 2013
When working on a shared network with limited bandwidth, it’s sometimes nice to be able to keep listening to Spotify without ruining your co-workers’ Internet connections.
Spotify is P2P (the desktop app is, anyway), so you both receive data from other Spotify users and transmit it to them. Blocking the outgoing traffic entirely would be un-neighbourly (probably also a violation of TOS), and might even prevent streaming from working altogether.
Fortunately, as I just discovered, ipfw allows you to add a pipe to a range of ports, all of which you can then throttle to a certain data transfer rate. It’s a shotgun approach, and will slow down any other services that are trying to send data over those ports as well, but since Spotify seems to stick to the range 10000 to 80000, and I rarely if ever run anything of consequence on those ports, this approach works for me.
sudo ipfw add pipe 1 ip from any to any out dst-port 10000-80000 sudo ipfw pipe 1 config bw 8KBytes/s
by Luke Rodgers on April 24, 2013
ExtJs Grid provides a nice, quick way to build a UI that can handle a lot of tabular data and support the operations you’d typically like to perform on that data (sorting, filtering). It can be hard to style, but is great for whipping up admin CRUD functionality.
Ext.grid.Panel does in fact expose the requisite properties and methods to accomplish this, but they’re a bit hard to track down, and not part of the official public API. This gist, GridFilteringExtensions.js, adds some methods to Ext.grid.Panel that make this easier. It probably won’t accomplish everything you want to do, and hasn’t been tested in a wide variety of situations, but it was exactly what I needed, and may be a good base from which to build.
Note that before manually setting a filter for the first time, you’ll need to call `createFilters`.
by Luke Rodgers on April 16, 2013
I think they are reasonably complete. Bug reports, pull requests welcome.
by Luke Rodgers on March 15, 2013
A beautiful and bitter poem by Theognis, addressed to his beleaguered young lover Kurnos.
I give you wings. You’ll soon be lifted up
Across the land, across the boundless crests
Of ocean; where men dine and pass the cup,
You’ll light there, on the lips of all the guests,
Where lithe, appealing lads will praise you, swelling
Their song to match the piper’s sweet, shrill tone.
At length, my boy, you’ll enter Hades’ dwelling,
That black hole where departed spirits moan,
But even then your glory will not cease,
Your well-loved name will stay alive, unworn;
You’ll skim across the mainland, over Greece,
Over the islands and the sea, not borne
By horses, Kurnos; you’ll be whirled along
By violet-crowned maids, the Muses; yours
Will be each practiced singer’s finest song,
As long as light exists and earth endures.
I give you this, for what? To be reviled–
To be betrayed and lied to, like a child.
by Luke Rodgers on March 14, 2013
In order to reproduce what this will look like in your app/website it can be helpful to artificially slow down your connection to Typekit. This can be accomplished via ipfw (no guarantee that these commands will work exactly as below for other unix variants).
First get the IP address of use.typekit.com by pinging it (for me it is currently 22.214.171.124). Then:
sudo ipfw add pipe 1 ip from 126.96.36.199 to any
sudo ipfw pipe 1 config bw 80kbit/s plr 0.05 delay 50ms
Play with the values 80 to change bandwidth, 0.05 to change packet loss ratio (you can just remove this as well), and 50 to change latency.
When you’re done (note that this will flush all existing ipfw rules):
sudo ipfw flush
by Luke Rodgers on February 3, 2013
# size commit date 439323 d4d09e047d50388180a1e317efc61af5d8961275 20130201 439323 fd30e151e35efba1bda65488e621c7338895542e 20130130 439241 6ce650d7e97add955b7cd07150732890c0edaf49 20130129 439241 3c1d2aec69f874926965843800163be71ec5f376 20130128
If the name of the file stays the same, it turn out this is pretty simple. The following git command will show the size of the file for the commit in question:
git ls-tree -r -l <COMMIT> <PATH>
So we can do something like
git ls-tree -r -l HEAD~$COUNTER compiledjs.min.js
in a bash script and increment $COUNTER as much as we want, grabbing the file size with some ugly use of tr and cut, e.g:
git ls-tree -r -l HEAD~39 compiledjs.min.js | tr -s ' ' | tr '\t' ' ' | cut -d ' ' -f 4
But if the name of the file changes across commits, as it will if you are tagging it with a date or SHA1 for cache-busting, this approach won’t work. The approach I came up with, which is hacky, involves creating and deleting temporary branches based on HEAD~1, HEAD~2, etc., and getting the requisite date, file size, and commit info by pattern-matching on the name of the file in question.
Shell script to accomplish this, along with some basic gnuplot commands to plot the output, here: https://gist.github.com/4700556