Sometimes I find myself needing to run a few command on a remote machine. I’ll usually ssh into the box and hack away. But, there is an a alternative. You can run command over SSH!

In the example below, I wasnted to keep the configuration files in the git repository, but wanted a quick and dirty way to deploy the changes.

The first half of the script copies files, using scp, from the local machine to the remote server.

The second half of the script (starting on line 12) runs commands from the remote machine. The files are moved to the correct locations and nginx and haproxy are restarted.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

#!/bin/bash

PEM=id_rsa.pub
HOST=ec2-54-234-130-49.compute-1.amazonaws.com

scp -i ~/.ssh/$PEM ./surrogate_pop.conf ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./haproxy.cfg ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./traffic_cop.lua ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./allowed_domains.lua ubuntu@$HOST:/tmp

## These are executed on the remote host
ssh -i ~/.ssh/$PEM ubuntu@$HOST 'bash -s' <<EOF
sudo mv /tmp/traffic_cop.lua /usr/share/nginx/traffic_cop.lua
sudo mv /tmp/allowed_domains.lua /usr/share/nginx/allowed_domains.lua
sudo mv /tmp/surrogate_pop.conf /etc/nginx/sites-enabled/surrogate_pop.conf
sudo service nginx restart

sudo mv /tmp/haproxy.cfg /etc/haproxy/haproxy.cfg
sudo service haproxy restart
EOF

Recently, I’ve been excited by serverless technology. I began using the serverless framework for code boilerplate and deployment. After some time using the framework, I began feeling pain. Serverless is an excellent project, but it’s moving very fast. For example, The framework uses cloudformation for resource dependencies such as dynamoDb, ApiGateway, roles and permissions (to name a few). Cloudformation is also moving very fast. Support for ApiGateway was added to cloudformation on April 18th, 2016. As new features are added to cloudformation, you’ll be stuck waiting for serverless to implement features for feature parity. I’ve started using cloudformation direclty and relying on bash scripts for deployment. I’m quite happy with the results.

Cloudformation stack

Once we have a cloudformation template, the AWS cli provides us with everthing we need. Using the AWS CLI we can create the stack like so.

1
2
3
4
5
6

$ aws cloudformation create-stack \
  --stack-name hello-world \
  --template-body file://./cloudformation.json \
  --capabilities CAPABILITY_IAM && \
  aws cloudformation wait stack-create-complete \
      --stack-name hello-world

The first command fires off an async create request to AWS. The second command tells our shell to wait for stack creation to complete.

After that’s complete, we’ll have created a few resources in AWS. =) Next, we’ll need a way to deploy.

Deployment

We have a few tasks for a complete deployment. We should seperate out the lambda depoyment and the ApiGateway deployment, but in this case I did not.

• Update Lambda Code - Install any dependencies, zip our code, and upload it.
• Publish a Version - From the latest version, It will tag a copy.
• Update the alias - Our lambda is pointed to an alias. This will point the lambda to our new version.
• deploy ApiGateway - Any changes we make to ApiGateway requires a deploy.

The script takes two args. The api-gateway-id and the function-name.

1

$ ./deploy.sh abc123 hello-word

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

#!/bin/bash

apiId=$1
functionName=$2
profile=personal

YELLOW='\033[0;33m'
WHITE='\033[0m' # No Color

function zipLambda {
  say "Zipping files." && \
  rm -rf target && \
  mkdir -p target && \
  cp -r *.js package.json target/ && \
  pushd target && \
  npm install --production && \
  zip -r "${functionName}.zip" . && \
  popd
}

function say {
  printf "\n${YELLOW} $@ ${WHITE}\n"
}

function updateLambdaCode {
  say "Uploading new lambda code." && \
  aws lambda update-function-code --function-name $functionName --zip-file "fileb://target/${functionName}.zip" --profile $profile
}

function publishVersion {
  say "Publishing a new version." && \
  aws lambda publish-version --function-name $functionName --profile $profile
}

function updateAlias {
  version=$(aws lambda list-versions-by-function --function-name $functionName --profile personal | grep Version | tail -n 1 | cut -d '"' -f 4) && \
  say "Updating the alias to version ${version}." && \
  aws lambda update-alias --function-name $functionName --function-version $version --name prod --profile $profile
}
function deployApiGatway {
  say "Deploying to Api Gateway." && \
  aws apigateway create-deployment --rest-api-id $apiId --stage-name v1 --profile $profile
}
printf "\n🚀🚀🚀 SHIP IT!!! 🚀🚀🚀 \n\n"
zipLambda && \
  updateLambdaCode && \
  publishVersion && \
  updateAlias && \
  deployApiGatway

Everyone loves clever, right!?. Well, DONT! A clever solution is technical debt.

Any fool can write code that a computer can understand. Good programmers write code that humans can understand. – Martin Fowler

So why do we do this? I belevie we enjoy knowing something that confuses others. It makes us feel smart. But please avoid this. Avoid this for your app, for your teammates, and for your future self.

Rails 4 brought us ActiveModel::Model. It provides a light weight interface that’s similar to an ActiveRecord::Base model.

for example, I can create a Person class like so.

1
2
3
4
5
6
7
8
9
10

class Person
  include ActiveModel::Model
  attr_accessor :name, :age

  validates :name, true

  def save
    ## Do cool stuff here...
  end
end

1
2
3
4
5
6
7
8

Loading development environment (Rails 4.2.1)
irb(main):001:0> p = Person.new age: 21
=> #<Person:0x007fc6367193b0 @age=21>
irb(main):002:0> p.valid?
=> false
irb(main):003:0> p.errors
=> #<ActiveModel::Errors:0x007fc638000a68 @base=#<Person:0x007fc6367193b0 @age=21, @validation_context=nil, @errors=#<ActiveModel::Errors:0x007fc638000a68 ...>>, @messages={:name=>["can't be blank"]}>
irb(main):004:0>

This is great for instances where you don’t need a full database backed Active Record model. I’ve used them for form objects and in controllers where I have complex logic.

You can think of these as higher level abstractions above your ActiveRecord classes. Also, be conscious of the dependancy direction. An ActiveModel model can depend on an ActiveRecord model but your ActiveRecord models shouldn’t depend on an ActiveModel model.

Here’s a more involved example. Lets say I have 2 ActiveRecord classes Org and User

1
2
3

class Org < ActiveRecord::Base
  validates :name, presence: true
end

1
2
3

class User < ActiveRecord::Base
  validates :first, :last, presence: true
end

Now I’ll create an ActiveModel model (non database)

Notice the validates_each method… Its going to check each of the ActiveRecord objects and let them raise up any errors to the Signup class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

class Signup
  include ActiveModel::Model
  attr_accessor :first, :last, :name

  validates_each :user, :org do |record, attr, value|
    unless value.valid?
      value.errors.each { |k,v| record.errors.add(k.to_sym, v) }
    end
  end

  ## must return boolean
  def save
    if valid?
      org.save && user.save
    else
      false
    end
  end

  private

    def org
      @org ||= Org.new(name: name)
    end

    def user
      @user ||= User.new(first: first, last: last)
    end
end

Awesome right!!

So why do all this? Well, the single responsibilty states that every class should have responsibility over a single part of the functionality provided by the software, and that responsibility should be entirely encapsulated by the class. ActiveRecord is responsible for persistence to the database. This will keep our classes with a narrow focus and allow us to refactor and create more use cases in the future. I think it’s a win. I find this strategy is generally good for one directional workflows such as signup or in a shopping app cart checkout.

Occasionally I’ll see things like this in a code base.

1
2
3
4
5
6
7

def api_host
  if Rails.production?
    "http://prod.fake.api.url"
  else
    "http://stag.fake.api.url"
  end
end

I try to avoid writting methods like this. Rails provides a nice way to set environment specific variables.

http://guides.rubyonrails.org/configuring.html#custom-configuration

config/environments/staging.rb

1

config.api_host = "http://stag.fake.api.url"

config/environments/production.rb

1

config.api_host = "http://prod.fake.api.url"

So now you can refactor the method to this.

1
2
3

def api_host
  Rails.configuration.api_host
end

I’ve been playing around with Public-key cryptography or asymmetric cryptography. With revaltions from Eric Snowden on programs like PRISM, I feel that encryption is more important than ever.

From Wikipedia

Public-key cryptography is a class of cryptographic algorithms which requires two separate keys, one of which is secret (or private) and one of which is public. Although different, the two parts of this key pair are mathematically linked. The public key is used to encrypt plaintext or to verify a digital signature; whereas the private key is used to decrypt ciphertext or to create a digital signature.

There are two main uses for public key crypto.

1. Public key encryption. A message is encrypted with a recipient’s public key. The message can only be unencrypted by the holder of the private key.

2. Digital Signature. A message is signed with a the senders private key to generate a cryptographic hash. Using the public key, one can verify the message was indeed sent with the coresponding private key and can ensure the message has not been altered.

Keys

First, let’s generate some keys. From the private key we will also generate the public key. Remember the private and public key are mathematically linked. This is important.

1
2
3
4
5
6

[1] pry(main)> require "openssl"
=> true
[2] pry(main)> private_key = OpenSSL::PKey::RSA.new(2048)
=> #<OpenSSL::PKey::RSA:0x007ff03b379c30>
[3] pry(main)> public_key = private_key.public_key
=> #<OpenSSL::PKey::RSA:0x007ff03c3330b8>

Great. Now we have a key pair. We can write these keys to disk if we like.

1
2
3
4
5
6
7
8

## write the private key to disk.
File.open("~/.ssh/private_key", "w+") do |f|
  f.write private_key.to_pem
end
# write the public key to disk.
File.open("~/.ssh/public_key.pem", "w+") do |f|
  f.write public_key.to_pem
end

We can read the key in from key like so.

1

key = OpenSSL::PKey::RSA.new(File.read("~/.ssh/private_key"))

Public key enctryption

We’ll be encrypting a message with the public key. So it can only be unenctrypted by the owner of the private key.

1
2
3

## Use the public key to encrypt a message
[4] pry(main)> encrypted_data = public_key.public_encrypt("Some private data is here.")
=> "\x9B\f\xD2\"\x02l\x0E\xFB\x18\xFA\xA2\xCC;\x9F\xC2r\xFA\x88\x94\\\xDBZY\x964\xE3\x13i\xDB\xB5\xB4<\x8E\xEC\xB8\xAB\"a\xE1\xE0\x01\x8Cb\xEC\xAE\x86\xC0!\x18\xC8\xA9\xDEB\e^\xCCe\x11\xDC\x19\x81\xDC\x98\x1EX\x11\xD6\"bUJ\xB0B\xED\x96\x03\x9F}\xF7I\xFD\x9F\x04\x04\xF4Q\xA7H\x9C\b\xC5\a\xE1\xA4\x02kx\xF46A% \xEB\xEB: \xD1\xA2\xFF\xA4$\xBDP\x9FR\xE9\xD3\x9F\r>\x8E5\xD9\x10\xE8\x9DYd\e`\x8E\xE0\xBD\xE5\\\x02\xF6\x83\v\x15\xD4m\x03\xCDJ\x9A*\xA6\xF5S9\xD2M\x10\xD5 7\x92\xCE\xF0\xB9<\xAFe\x9A\xAE\x8E`\xADW\x93\x91;HZ\xEB\xD8\xFF\xDE\x02\x10m\x9D\xF2*\tL2\xE2`K\xC8\x0E)O\x86\xDA\xAFk4\xD0\rN\xF3%}\xEAM\x15he\xDF\x0F\xC6\xE5\xAA$\x19\x03V\xBE\x0E~\x8ELv\x9C\xC7\xB4o\xA13\xE2\xDE\xE2t!]-\xC8\x88\xAF\xE3\xD5\xFD`T\xD9\xCDs\xFF\xAE(<\xBD\xB9"

This will result in some unreadable binary data. Next we’ll decrypt this data with the private key.

1
2
3

## Use the private key to decrypt
[5] pry(main)> private_key.private_decrypt(enc)
=> "Some private data is here."

Nice!

The public key can encrypt but cannot decrypt the message. This enctryption only works in one direction. Only the private key can decrypt that message. You can also encrypt messages with a private key. Those messages can be unenctrypted with the public key and the private key.

Asymmetric public/private key encryption is slow and victim to attack in cases where it is used without padding or directly to encrypt larger chunks of data. Typical use cases for RSA encryption involve “wrapping” a symmetric key with the public key of the recipient who would “unwrap” that symmetric key again using their private key.

NOTE - The enctrypted messages are in binary format. If we want to encode the data for transmission accoss the net, we’ll want to use something like base64.

1
2
3
4

[6] pry(main)> require "base64"
=> true
[7] pry(main)> enc_string = Base64.encode64(encrypted)
=> "iRBt6WHFYP2sn2Qv+qs16js/EJqBGaTWyxUH7iI/aj3UEw1oUHHbrFs/705W\nP+8dJ77p5gAaBpS/spCYLu/strU3uN06DTOh3neTcyDQrpIL5Zqs0Gl6/76m\nOQFGi18khnwWPAyW4+uVcZiQmZU9M0tlWywwNlkVoKAwkFlwkYF07YZazfCY\nMrAoQ6nusfjqjfU7HQeQKSCnMrBkzInsqan0PUm+UuGCMbxpQMdPA1de2nHB\neMs7OR7Pd5q5T93z240Iacjtwo/CV3Tcr1EyrfCcx05Jp4FKi9DPJf33asPx\noJ7J2XNa3QXqEbisMGT/b+6QZDm/LbfZXKuCIDYOjA==\n"

This works for small messages but if you try and encrpt a string larget than your key… you’ll hit an errors. You could break up the string into smaller chunks and encrypt each chunk individually. But this is not secure. This is what cypher block chaining is for. Below is an example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

cipher      = OpenSSL::Cipher.new 'AES-128-CBC'
cipher.encrypt
iv          = cipher.random_iv
pwd         = 'tRiSsmiTp'
salt        = OpenSSL::Random.random_bytes 16
iter        = 20000
key_len     = cipher.key_len
digest      = OpenSSL::Digest::SHA256.new

key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest)
cipher.key = key

File.open("enc-text.txt",'w') do |enc|
  File.open("./plain.txt") do |f|
     loop do
       r = f.read(4096)
       break unless r
       enc << cipher.update(r)
     end
     enc << cipher.final
  end
end

Digital Signature

Asymmetric digital signatures is a great way to verify integrity and authenticity of data. Create a keypair, send the public key to your receivers, and use this method to create a digital signature. By combining the data and the public key, you can verify that the signature was created by the owner of the private key.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

require "openssl"

data = "A small brown fox."


digest = OpenSSL::Digest::SHA256.new
# To list available digests:
#OpenSSL::Digest.constants

signature = private_key.sign(digest, data)

public_key.verify(digest, signature, data)
# => true
public_key.verify(digest, signature, data + "altered")
# => false
public_key.verify(digest, "altered" + signature, data)
# => false

More

Using openssl library directly we can encrypt a file.

1

openssl aes-256-cbc -a -e -in source-file.txt -out enc-file.txt -k private_key

And to decrypt the file.

1

openssl aes-256-cbc -d -a -in enc-file.txt -out unenc-file.txt -k private_key

http://ruby-doc.org/stdlib-2.0/libdoc/openssl/rdoc/OpenSSL.html#label-PBKDF2+Password-based+Encryption

I’m in and out of servers all day. One thing I’ve found helpful is a ssh config file. Create a file and place it ~/.ssh/config. You can add as many entries as you like.

1
2
3
4

Host s3-rsync
  HostName ec2-54-203-43-190.us-west-2.compute.amazonaws.com
  User ec2-user
  IdentityFile "~/.ssh/private-key-location"

Host: A friendly name/handle which you’ll use.

HostName: The remote server address you’ll be connecting to.

User: The user you want to connect as.

IdentityFile: The private key to authenticate with.

Once its all setup you can connect by typing: ssh s3-rsync

This is the same as: ssh -i ~/.ssh/bs2-deploy ec2-user@ec2-54-203-43-190.us-west-2.compute.amazonaws.com.

Rails has taught me some really good best practices. Let’s look at a default rails create action

1
2
3
4
5
6
7
8
9
10

def create
  @post = Post.new(post_params)
  respond_to do |format|
    if @post.save
      format.html { redirect_to @post, notice: 'Post was successfully created.' }
    else
      format.html { render :new }
    end
  end
end

I try not to deviate too much from this and nor should you. If @post.save returns true. return a 201 status code. If @post.save returns false re-render the :new action. So you can let the user fix the form.

Often I will see things like this.

1
2
3
4
5
6
7
8

def create
  if params[:something]
    @post = Post.create(post_params)
  else
    @post = Post.create(other_params)
  end
  redirect_to posts_path
end

At first glance this looks clean. But if you look closer. There is a subtle problem here. If the post object fails validation and fails to create, The action redirects to the index action where all new instaces are creating. The @post object has lost any knowlege of the params and it will not get to oportunity to display the errors from the object. You’ll have to resort to things like flash[:notice] = "@post.errors.full_messages.join()"… ugh.

Active record gives us a nice interface for displaying errors on an object.

1
2
3

class Post < ActiveRecord::Base
  validates :title, presence: true
end

1
2
3
4
5
6
7
8

irb(main):001:0> p = Post.new
=> #<Post id: nil, title: nil, body: nil, created_at: nil, updated_at: nil>
irb(main):002:0> p.valid?
=> false
irb(main):003:0> p.errors
=> #<ActiveModel::Errors:0x007fe0ca9e4b68 @base=#<Post id: nil, title: nil, body: nil, created_at: nil, updated_at: nil>,
@messages={:title=>["can't be blank"]}>
irb(main):004:0>

When you call @post.save OR @post.valid? errors are added to the @post object so they can be displayed within the form.

Digging even deeper… Check out ActiveModel::Model. It will give you this same object interface without needing database backed model. Powerful stuff.

Lately I’ve been writing a lot of angularjs. At my work, we had a great debeate.. which client side framework should we use. It came down to angularjs or ember. Ultimately, we choose angular. I’m happy we did. Angularjs is awesome. I would have been just as happy to settle with ember. Two way binding is a game changer.

Rails is great, BUT, I feel that, in any rails app, you end up creating interfaces that are a reflection of your database. Adding a client side framework, you’re free to create interfaces that are decoupled from your backend.

Lately I’ve been doing a lot of server administration. It can be a lot of fun automating systems. One thing I’ve found super valuable is to setup a “message of the day” file. This will display immediately after login and can be found in /etc/motd. I like to put notes/documentation here. For example, I’ll put in how to restart redis, how to tag a deploy, or the path to the app.