Recently I received a request to encrypt some part of data that was already in my client’s app. I don’t want to share too many details, so let’s say it was few columns of an existing table that already contained some data.
So my task was to enrypt all existing records, delete non-encrypted data and save all future records as encrypted. Sounds cool!
What I needed was a simple ruby gem that would help me with all above tasks. I found Lockbox or - the Lockbox found me! I’ve read about it some time ago in RubyWeekly newsletter. It offers migrating existing data, encrypting columns, decrypting fields on the fly if you need to display them somewhere in the views, etc.
Since our app is used 24/7 we didn’t want to put it in the maintenance mode, we had to split update into few steps without causing any downtime. I came up with this solution:
- Add Lockbox to gemfile, add new columns to persist encrypted data, update model to migrate existing data
ignored_columnsattribute to model (to remove columns in next step without any downtime)
- Remove non-encrypted columns from table
This required 4 pull requests creation, 1 task execution and zero downtime :)
Execution I started by adding
gem 'lockbox', '~> 0.4.6' to the Gemfile and generating Lockbox master key.
This key had to be added to all secrets (in dev, staging, production and test environments). I also created
lockbox.rb file inside
After that Lockbox was ready to go, so I could start work on encryption of my desired model. Let’s say that I wanted encrypt our clients’ correspondence data. Inside
Client::Address model, I had to encrypt following fields:
models/client/address.rb I had to add:
class Client::Address < ActiveRecord encrypts :street, :city, :postal_code, :country, :phone_number, migrating: true end
Notice last part of it -
migrating: true. This is crucial for next step - migrate existing data. But before that I had to create new columns for encrypted data. So I came up with following migration:
def up add_column :client_address, :street_ciphertext, :text add_column :client_address, :city_ciphertext, :text add_column :client_address, :postal_code, :text add_column :client_address, :country, :text add_column :client_address, :phone_number, :text change_column :client_address, :street, :string, null: true change_column :client_address, :city, :string, null: true change_column :client_address, :postal_code, :string, null: true change_column :client_address, :country, :string, null: true change_column :client_address, :phone_number, :string, null: true end
At this point no
null: false constraint could take place - adding
not_null columns to existing records would paralyze app. Also
null: true had to be added to existing columns - from now on we would save new records only to
After successful migration, there was only one thing missing. Data migration itself! Thanks to Lockbox, this is done by running simple command in rails console. After merging above changes, I had to ssh to production environment and run
At this point all existing data was migrated to
_ciphertext columns. But I needed more - I needed to remove non-encrypted columns with sensitive data saved in plain text!
First of all I removed
migrating: true from
models/client/address.rb model - this was no longer needed. Next thing was migration:
def up change_column :client_address, :street, :text, null: false ... end
_ciphertext columns I added
not_null constraint, since we’re already saving records to that columns and none of them was empty anymore. And the most important part of this pull request was adding
ignored_columns to Address model. Since in next PR I wanted to remove all unnecessary columns this was crucial at to add this at this point.
class Client::Address ... ignored_columns: %w[street city postal_code country phone_number] ... end
This part is only about removing non-encrypted columns. So it included only this migration:
def up remove_column :client_address, :street remove_column :client_address, :city remove_column :client_address, :postal_code remove_column :client_address, :country remove_column :client_address, :phone_number end
ignored_columns this could be deployed without causing any downtime to app. From this point there were no decrypted clients’ address data in our database.
Last step was to remove
ignored_columns attribute from the
Encryption was definitely worth adding and considering fact that this update was deployed at Friday after 2a.m. with no issues, this shows that process is very easy. At least on that set of data that I had to encrypt.