the other dell

Pushing Heroku's buttons

Nov 5, 2015

I’m re-making Three Word Poems to help me learn enough Rails to be useful. I’ve had a version of this sitting on my development machine for some time. It’s not ready for production.

I’ve reached Chapter 7: Sign Up of Michael Hartl’s The Rails Tutorial. In this chapter students add a sign-up form to their tutorial project and learn about validation. They also configure their app to use SSL and Puma in production. I have more spare time at the moment so I’m willing to face the deployment issues I didn’t want to look at earlier. This will probably lead to yak shaving.

Problem one: No remote hosting

The tutorial invited me to git push heroku, which would have sent the code to the remote server and kicked off a build and deployment process. Because I work on personal projects very slowly and am easily side-tracked, I forgot I’d not even begun to create a Heroku application for the project. This is easily remedied:

heroku create
git push heroku

That set up the hosting and began a deployment. However, the build failed.

Problem two: wrong database

The deployment showed this error:

remote: An error occurred while installing sqlite3 (1.3.10), and Bundler cannot remote: continue. remote: Make sure that gem install sqlite3 -v '1.3.10' succeeds before bundling. remote: ! remote: ! Failed to install gems via Bundler. remote: ! remote: ! Detected sqlite3 gem which is not supported on Heroku. remote: ! https://devcenter.heroku.com/articles/sqlite3 remote: ! remote: remote: ! Push rejected, failed to compile Ruby app

The deployment was proactively rejected. This is a good thing: I wont be wasting cycles on a Dyno running a fauly app. But it’s not I want to happen; I want a functioning application and I want to get into incremental deployments.

For me, the key part was “Detected sqlite3 gem which is not supported on Heroku.”. The database in Heroku’s default stack for Rails apps is Postgress. My application was using the default Gemfile created by rails new myapp, with only minor additions. Thus the sqlite3 gem was defined for all environments. A better solution (for now) is to use sqlite3 in local development and Postgress on “production”[1][2].

Some tinkering with my Gemfile, lead to this configuration:

group :development do
	gem 'spring'
	gem 'sqlite3'
	gem 'byebug',      '3.4.0'
end

group :production do
	gem 'pg',             '0.17.1'
	gem 'rails_12factor', '0.0.2'
	gem 'puma',           '2.11.1'
end

I’ll skip explanation of all the choices behind this config, but I suspect it’s easy enough to work out. Thankfully, also, it allows the application to be built and deployed.

Problem three: application error

However, visiting the remote URL leads to a Heroku-styled “Application Error” message. I discover, through some searching I can’t now reconstruct, that heroku logs does what you’d hope it would. The most obvious error in the logs is this 503:

2015-11-05T22:04:26.292214+00:00 heroku[router]: at=error code=H10 desc=”App crashed” method=GET path=”/” host=my-app-XXXX.herokuapp.com request_id=0191567f-3ea1-421f-8fe2-5e29e23d886c fwd=”87.113.223.41” dyno= connect= service= status=503 bytes=

However, looking up the logs, there’s some pre-crash logs:

2015-11-06T12:18:39.683157+00:00 heroku[web.1]: Starting process with > command bundle exec puma -C config/puma.rb 2015-11-06T12:18:42.145820+00:00 app[web.1]: [3] Puma starting in cluster > mode… 2015-11-06T12:18:42.145834+00:00 app[web.1]: [3] * Version 2.11.1 (ruby 2.0> .0-p645), codename: Intrepid Squirrel 2015-11-06T12:18:42.145835+00:00 app[web.1]: [3] * Min threads: 5, max > threads: 5 2015-11-06T12:18:42.145838+00:00 app[web.1]: [3] * Environment: production 2015-11-06T12:18:42.145856+00:00 app[web.1]: [3] * Process workers: 2 2015-11-06T12:18:42.145880+00:00 app[web.1]: [3] * Preloading application 2015-11-06T12:18:44.046287+00:00 app[web.1]: PG::UndefinedTable: ERROR: > relation “categories” does not exist 2015-11-06T12:18:44.046291+00:00 app[web.1]: LINE 1: SELECT > “categories”.* FROM “categories” ORDER BY “categor… 2015-11-06T12:18:44.046292+00:00 app[> web.1]: ^ 2015-11-06T12:18:44.046293+00:00 app[web.1]: : SELECT “categories”.* FROM > “categories” ORDER BY “categories”.”id” ASC LIMIT 1 2015-11-06T12:18:44.046458+00:00 app[web.1]: [3] ! Unable to load > application: ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: > relation “categories” does not exist…

This looks like the database has not been initialised. If I were preparing a local deployment to develop in, I’d run this:

bundle exec rake db:migrate
bundle exec rake db:seed

The equivelent for the Heroku deployment is this:

heroku run rake db:migrate
heroku run rake db:seed

That begins well, but fails half-way through the migration.

Problem four: failing migrations on remote

The first error is this:

Migrating to RemoveImageIdFromPoems (20150418174423) rake aborted! NameError: uninitialized constant RemoveImageIdFromPoems /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/active_ support/> inflector/methods.rb:238:in const_get' /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/ active_support/> inflector/methods.rb:238:in block in constantize’ /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/ active_support/> inflector/methods.rb:236:in each' /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/ active_support/> inflector/methods.rb:236:in inject’ /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/ active_support/> inflector/methods.rb:236:in constantize' /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.6/lib/active_ support/> core_ext/string/inflections.rb:66:in constantize’

I need to work out why the RemoveImageIdFromPoems migration might fail.

This thread seems to describe a similar problem. I try the brute force solution:

hekoku pg:reset

But get this refusal:

% heroku pg:reset ! Usage: heroku pg:reset DATABASE ! Must specify DATABASE to reset.

I don’t know what the database will be called. It’s not listed in the “db/schema.rb” file. I look in “config/database.yml”.

default: &default
  adapter: sqlite3
  pool: 5
  timeout: 5000

# ...

production:
  <<: *default
  database: db/production.sqlite3

This definition for the production database seems wrong: it’s not referencing postgress at all. I guess I have more than one problem to fix!

I changed the database.yml to include this snippet:

production:
  adapter: postgresql
  encoding: unicode
  pool: 5

At first I got this wierd error when pushing to Heroku:

remote: —–> Preparing app for Rails asset pipeline remote: Running: rake assets:precompile remote: rake aborted! remote: IndexError: string not matched

Eventually I found a typo in database.yml, and correcting that allowed the application to deploy. However, now I get a 500 error when going to the URL, rather than a 503. The 500 seems more general than the 503, but somehow makes more sense: I’m not really expecting the application to work at the moment. heroku logs shows this:

2015-11-09T16:34:51.342239+00:00 heroku[router]: at=info method=GET path=”/” host=my-app-host.herokuapp.com request_id=e85ffc3c-f7f3-4dfb-a9b6-85edc3808a6d fwd=”87.113.223.41” dyno=web.1 connect=1ms service=6ms status=500 bytes=1714 2015-11-09T16:34:51.341117+00:00 app[web.1]: Completed 500 Internal Server Error in 0ms 2015-11-09T16:34:51.337896+00:00 app[web.1]: Started GET “/” for 87.113.223.41 at 2015-11-09 16:34:51 +0000 2015-11-09T16:34:51.340702+00:00 app[web.1]: Processing by CategoriesController#show as HTML 2015-11-09T16:34:51.341991+00:00 app[web.1]: 2015-11-09T16:34:51.341993+00:00 app[web.1]: NoMethodError (undefined method id' for nil:NilClass): 2015-11-09T16:34:51.341994+00:00 app[web.1]: app/controllers/categories_controller.rb:6:in show’

This is pointing out a problem in a specific view (or perhaps in a Controller method): CategoriesController#show. However, it occurs to me that the database might still not be ready. I try heroku run rake db:migrate again, with this response:

Running rake db:migrate on my-app-host… up, run.4081 Migrating to RemoveImageIdFromPoems (20150418174423) rake aborted! NameError: uninitialized constant RemoveImageIdFromPoems

Seems the database is still not initialised. Perhaps I forgot to run the migrations.

After some almost random futzing about, it dawns on me to actually look in the particular migration file. I notice the filename is “20150418174423_remove_image_id_from_poems.rb”, but the migration class name is RemoveImageIdFromPoem, rather than the RemoveImageIdFromPoems identified in the error message. Experimentally I add an “s” to the end of the class name, watch it run locally and then risk pushing to Heroku. Amazingly, this allows the migrations to run on Heroku! I can’t explain what is different between the local and remote task runner that would cause this issue, at this point.

Problem five: Seed data

It actually didn’t work first time. I figured I’d have to run rake db:seed on Heroku as well as rake db:migrate, so I checked it still ran locally after changing the migration name. It migrated fine, but failed to seed the database. Going through the earlier chapters in The Rails Tutorial, I’d added password validation to the Poets model, but had not added passwords to the seed data. I added these to get the tasks to run[3].

After running bundle exec rake db:seed locally, I checked whether the site still ran. It didn’t. This lead me to discover something broken in my image uploading (because the seed data also uploads dummy images from Lorempixel). There’s also a problem with the seeding in general. The symptoms are:

  1. going to the root URL generates an error around category/poem associations
  2. typing Poem.all.count in Rails Console returns “0” where it should be 10.

The image uploading uses CarrierWave, which in turn uses RMagick, which in turn uses imagemagick. I’ve written elsewhere about problems with some of these. That earlier post describes how I reinstalled imagemagic and rmagic. On reflection, I wonder if the imagemagick version “homebrew/versions/imagemagick-ruby186” was really the correct choice: my local ruby version is 2.1.3p242 and the one on heroku is 2.0.0p645. These should probably match, but more significantly (for the current issue) they’re both a major version ahead of 1.8.6.

I replaced ye olde imagemagick with the current one:

brew uninstall homebrew/versions/imagemagick-ruby186
brew install imagemagick

Next I removed all rmagick gems:

gem uninstall rmagick

There were two versions installed, so I removed them both. Then I re-installed the version required for the project.

bundle install

Before I did this, I remembered that I should probably restart the web server as well. Once the installation had completed and the server restarted, I found the application ran again (on my local development machine). Thank goodness!

Remarkably, at this point, I’m also able to push to heroku and have a successful deployment. Just to be sure, I ran:

heroku run rake db:reset

… which should totally regenerate the database. The command’s logs show several errors (such as lacking permission to drop the database), but the script keeps going. Once it finishes, I try the remote host root url and am amazed to discover the application is running. It seems to have the test/seed data, and will show me a couple of different views. The images are missing, but that’s not very surprising. I’ve read that hosting on Heroku isn’t set up for easy file uploads - ideally they’d go directly to external cloud hosting such as Amazon S3. I will cross that bridge later.

footnotes

  1. Heroku recommends using the same database in development and production. They’re not [alone][db3]. From a certain perspective, the Vagrant project and even dev-ops in general reflect many of these ideas.
  2. Well, I can pretend it’s “production”. Right now, it’s more of a production-like remote development environment that would probably get called “integration” in a commercial project.
  3. This revealed another issue: it’s insecure to put the seed users into the seed data - “seeds.rb” is under source control, hosted on Github, and therefore publically visible. I’ll have to change this, but that’s quite a different topic and will be dealt with and discussed elsewhere.