A blog about web development

Paul Sturgess

Reducing Siloed Knowledge With Great Documentation

Talk to most software engineers and they’ll tell you they’ve worked on projects with poor documentation and know first-hand that it causes frustration, bottle-necks and general inefficiency. Yet if you ask software engineers what they enjoy doing, writing documentation is either not on their list or it’s right down the bottom.

The tech team at Rail Europe places documentation very high up the list of priorities. This isn’t because we only employ people that enjoy writing documentation (although some of us clearly do! Myself most certainly included). Documentation is baked into the heart of our way of working.

However, I can assure you we haven’t all decided to write documentation in the name of productivity gains. The main benefit of documentation to remote teams is flexibility…

Avoiding blockers

Siloed information is the enemy of flexible remote work. We’re an internationally distributed team that believes in asynchronous communication. We want everyone to be able to work to their schedule, not a rigid 9-5. It’s no good if certain people have to be available and online all the time, to be around to answer questions and interject at any moment.

Once you trust each-other to manage your own time to get the work done, you have to optimise for it. Good clear documentation is the bedrock of this approach, as it enables people to be self-sufficient. Rather being blocked waiting for so-and-so to become available to answer a question, you can read the docs and make progress.

Onboarding

Not everyone gets an in-person induction, but there’s definitely plenty to read! Our onboarding documents are extensive and full of answers to questions that new starters ask. We know this because we encourage each new starter to update and enhance the documentation. If they can’t find the answer they’re after, they’re empowered to update the docs. We use Github’s wiki feature, everyone has write access from day one.

One of the main benefits of this is it avoids repetition and wasted time explaining the same processes and concepts. It also means the new starter can read and digest at their pace and doesn’t have to sit through multiple meetings taking copious amounts of notes.

Git as a living history

We love Ruby because it’s so expressive and the code itself is often self-documenting. But there’s a long-held tradition of well written commit messages in the Rail Europe (formerly Loco2) codebase. It’s a testiment to the engineers that pioneered the application, that the commits tell the story of not only what changed, but frequently why it changed, what alternative approaches were considered and other jewels of information around decisions that were made many years ago by people that no longer work at the company.

Because the example set was so strong, each new engineer has embraced it. Once code has been reviewed and is ready to be shipped, interactive rebasing and squashing is used so that “work in progress” commits are not included on the main branch. This means that “git log” reads like a well-written story.

This fits with our approach of continuous integration as each commit is atomic, which makes them easy to rollback if required (fortunately not often!).

How to use Git can be a hotly disputed topic within engineering teams. Our belief is that Github PR’s are great for discussing and reviewing changes and the “work in progress” commits make perfect sense in the context of the history of a PR. We don’t encourage squashing during a code review, but personally I like the idea mentioned on this blog post from Future Learn that just as we refactor code before merging, we should refactor our commits too.

On writing the commit messages themselves, the article A Note About Git Commit Messages basically sums it up for me.

Why does feature x work like that?

An adhoc form of documentation comes as a side-effect from our desire to work without meetings. This previous blog post I wrote goes into the detail, but to briefly summarise and mention one benefit… we openly discuss large new features before implementation using a Basecamp message board. This kind of blew my mind when I joined the company, as it meant I could go back in time and review the discussions that were had by the engineers right when they were discussing how certain features might work.

This kind of documentation is obviously not “curated” and designed as a long-term living document, like our wiki. But it can still provide valuable pieces of information that can otherwise be lost in the mists of time.

Summary

Documentaton comes in many forms and ultimately I think it’s one of the most under-rated and overlooked priorities that every software engineering team can benefit from.

Remote Working Without Meetings

As the COVID-19 pandemic continues, basically all software engineering teams are now working remotely. I’ve been part of a distributed software engineering team at Rail Europe for two and a half years, so I thought i’d reignite my blog to pass on some tips and experience.

In this post I want to focus on how we communicate with each other and make decisions, whilst rarely ever organising group video calls.

As I’ve emphasised in my previous blog posts, this isn’t about squeezing more productivity out of your team. It’s about trusting them and giving them the space and flexibility to arange their work day how it fits them. Scheduling loads of meetings takes away the freedom, autonomy and flexibility that is one of the major benefits of remote work.

At Rail Europe we prefer asynchronous communication by default and part of this approach is to keep all meetings to an absolute minimum.

Problems with meetings

Here are some issues with meetings (video or in-person):

  • Favours those that are best at thinking on their feet
  • Favours those with a more extrovert personality
  • Participants must stop work and be online at precisely the same time
    • If someone is off work, their opinion is not heard
    • Difficult with colleagues in different timezones
  • A one hour meeting for five people is really a five hour meeting
  • Often organised without a clear agenda
  • More effort to create a written record
  • Dominated by those in positions of power

Of course a good meeting facilitator can resolve many of these issues.

The alternative to meetings

At Rail Europe we prefer “long form” written discussions instead of meetings. We conduct these on Basecamp, but I’ve also used Github Team Discussions for the same purpose and it worked well.

We use it like a message forum – so someone starts a post on a specific topic and people reply to it.

These are great when you need to solicit the opinion of the wider team. Ultimately instead of calling a meeting, you describe in detail your proposal/idea/problem and select who you want to notify.

We also have certain long running threads that we use in place of “status update” meetings. So instead of scheduling a weekly catch up meeting – each time something significant happens you want to share with your team (or even the whole company). You post an update to a specific thread. Everyone stays informed, all the time without a meeting. This is critical for keeping people up-to-date even if they’ve had a week off on holiday (or maybe they’ve been off ill from COVID-19!).

Basecamp allows you to decide if you want to be emailed each time someone replies to a post or you can wait for the daily summary email. This fits into our idea that nothing is urgent.

For daily stand-ups, instead of holding a daily meeting, we post them asynchronously on Slack.

Benefits of writing things down

I must admit I was initially sceptical of this no meeting approach, I thought it would surely be less efficient than having a quick-fire back and forth video chat.

However, written long form discussions provide many benefits:

  • Doesn’t require everyone online at the same time
  • Read and reply in your own time without breaking your flow
  • Requires the author (would be meeting organisor) to think through their proposal and fully articulate it
  • Allows responses to be more considered (do some research first) rather than “knee-jerk”
  • If you’re off work, you can easily catch up later
  • Level playing field for participants, not just those that “shout loudest”
  • Can read historical discussions going back years to find out why something was decided
  • Discussion is transparent to anyone in the company
  • Easier to bring in people from the wider team to follow and contribute

Having participated in this type of team communication for a few years, I can now see that these longer, more considered discussions, often result in a decision that has more “buy-in” from the team. Anecdotally it also feels like we reach well thought-out solutions which, in the end save us time.

For all these reasons, we also avoid making team decisions in Slack and instead defer to a Basecamp post.

Don’t get me wrong, it’s not like we don’t ever have meetings. I have regular 1:1s and many tricky problems are best resolved over real-time discussions in Slack.

But for my direct team consisting of 5 engineers, 2 QA testers and a product manager – we haven’t had a meeting together in 2020 and it doesn’t look like we’ll have one any time soon.

Nothing Is Urgent – Asynchronous Communication

Lots of people have been thrown into remote working that aren’t used to it. I’m writing to share some tips that I hope will hopefully help reduce anxiety, stress and allow your team to work flexibly. As I mentioned before, this isn’t about squeezing more productivity out of your colleagues.

Previously I wrote about how distributed teams can adapt stand-up meetings to allow people to work more flexibly. Giving them space to adapt their working day around their personal situation.

I’ve had to shift my working hours to start in the afternoon, as I will now look after my son in the morning whilst my wife works. We switch roles in the afternoon. It’s pretty much like I’m working in a US time zone when most of my team is on CET or GMT. We do actually have some people on UTC-7, so I already have company.

Fortunately for me, the Rail Europe team is already geared up to communicate effectively even with people working completely different hours, whether time zone inflicted or not…

Asynchronous communication

Asynchronous commuinciation is based on the idea that “nothing is urgent”, especially when it comes to writing software. This might seem blindingly obvious when we’re in the middle of a global pandemic. In reality very few of us are doing anything that’s really truely urgent. That’s how you describe the heroic work of people trying to create a vaccine or key-workers on the front-line.

However, that’s not what I mean here. A communication style where “nothing is urgent” means we don’t interrupt our colleagues and expect immediate answers right now. We use asynchronous communication by default and we are mindful about the notifications we generate for our colleagues.

Jason Fried from Basecamp wrote this fantastic blog post on why Group chat is like being in an all-day meeting with random participants and no agenda. It informs much of our thinking.

I won’t pretend we are perfect at it and there is always an exception to the rule. But these are the specific (and default) behaviours we encourage amongst the tech team at Rail Europe:

  • Recognition that most questions do not need an immediate answer
  • There’s no expectation that your colleague/team is online all day
    • Muting of Slack channels or closing Slack entirely to avoid being interrupted is encouraged
  • Team decisions are not made in Slack
    • We use Basecamp for long-form discussions, so everyone can contribute when it suits them
  • @mention should not be used when someone is offline unless it’s critical
  • Respect Slack statuses such as “concentrating”, “lunch”, “gone for a walk”
  • Use the Lunch, Meeting or Phone status to avoid lots of pointless chatter in public channels
  • Task specific conversations that are relevant for project managers should go on Jira/Trello
  • Technical discussions about code should ideally happen on a Github PR
  • Consider an email over a Slack DM
  • Document all processes thoroughly
    • Good documentation allows people to be self-sufficient
    • Prevents “bus” factor

We keep meetings down to an absolute minimum, most are replaced by long-form written discussions. I’ll blog about this in more detail in the future.

Of course when I say “nothing is urgent” there are exceptions. An outage of our website and certain critical on-call alerts need to be responded to swiftly. And it’s not like we never communicate synchronously, it’s just that async is our default.

Adopting asynchronous communication really has many positive benefits on the well-being of your team:

  • Reduces notification fatigue
  • Reduces anxiety about being online all the time
  • Allows people to not feel guilty about taking breaks when they need to
  • Enables more flexibility for when people work
  • Because most communication and decisions are written down, if you’re off work it’s easy to catch up later
  • Everyone sees the value in great documentation and how it empowers them to work self-sufficiently
  • Conversations on Jira/Trello make it easy for anyone to understand the latest status of a task
  • Everyone gets longer periods of uninterrupted time to get into deep work

When it comes to remote work, asynchronous communication really is one of the most liberating things your team can adopt.

Stand-up Meetings for a Distributed Team

This global health emergency is horrendous and thus it looks like remote working will be the new normal for some time. I’ve been working remotely for Rail Europe for two and a half years, so I thought i’d reignite my blog to pass on some tips and experience.

Sadly this enforced remote working scenario has the potential to amplify all the difficult parts of remote working. Namely: dealing with isolation, fear that you need to “prove” you really are working, not separating home life from work life, amongst others.

To be clear, the ideas I want to share are not about squeezing out more productivity from your team. It’s about reducing stress and anxiety, trusting people and giving them room and flexibility to breathe. Remote working is great for all these things, but we distributed teams have to act and think differently to the typical co-located norms.

Anyway, I want to focus on one thing in this post…

Daily Stand-ups

These are really common in the tech industry. At Rail Europe we do them asynchronously. But what does that really mean?

  • Everyone writes their stand-up in a dedicated Slack channel
  • They post it (ideally) towards the beginning of their day
  • There’s no expectation that they all have to be posted at the same time
  • A stand-up message should contain 3 things:
    • What you worked on yesterday
    • What you plan to work on today
    • Any blockers
  • You’re encouraged to share how you are feeling or anything else effecting your work

This last point is important because, as a distributed team, you cannot rely on normal body language cues.

In theory this is a relatively simple change, that can have quite a dramatic impact, in a number of positive ways:

  • There’s no requirement for everyone to be online all at the same time:
    • This gives people flexibility to start work when it works for them
    • It’s especially benefitial if your team is in multiple-timezones
  • It’s less disruptive because you don’t need everyone to stop work at the same time.
  • For anyone off sick, or just not able to attend for any reason, they can easily catch-up on what happened while they were off
  • It’s massively more time efficient:
    • In-person stand-ups regularly get side-tracked off into details that are rarely relevant for the entire group
    • If someone outside your team wants a status update from you, they can read your stand-up
    • People can make their own decisions about whether they need to read every single update

If you need to follow up with someone on a specific point. We tend to use Slack threads or respond on the Jira Story/Trello card.

Ultimately the idea behind this approach is part of a wider communication strategy we have: To allow people to structure their working day how it works for them. To maximse and respect the amount of uninterupted time that we all have, so we can focus and stay in the flow.

Setup Webpacker Webpack-dev-server With Docker-compose

The rails/webpacker gem is a fantastic way to configure webpack for your Rails application. However, it has gone through quite a few different configuration options to get it working with Docker.

Initially you could set the host by using command line arguments, but as of version 3.0.2 and this Pull Request the command line argument support was removed in favour of environment variables. By setting various environment variables you can override all dev server settings defined in config/webpacker.yml.

So now my docker-compose.yml config looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
web:
    <<: *app
    command: /src/docker/web.sh
    ports:
      - "3000:3000"
    depends_on:
      - webpack
    environment:
      - WEBPACKER_DEV_SERVER_HOST=webpack

  webpack:
    <<: *app
    command: ./bin/webpack-dev-server
    ports:
      - 3035:3035
    environment:
      - WEBPACKER_DEV_SERVER_HOST=0.0.0.0

To install yarn and nodejs I added the following to my Debian based dockerfile:

1
2
3
4
5
6
7
8
9
10
RUN apt-get update && apt-get install apt-transport-https

RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
  && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list \
  && curl -sL https://deb.nodesource.com/setup_8.x | bash -

RUN apt-get update && \
  apt-get install -y \
  yarn
  nodejs

Debug Ruby Using Pry and Docker Compose

If you’re using Docker and docker-compose you probably found that by default you can’t debug your Ruby application with the binding.pry breakpoint.

However, if you comment out the command line for your web container. Then:

1
docker-compose up

Open a new tab and run:

1
docker-compose run --service-ports --rm web /the/command/you/commented/out

Then you will have interactive access to the command line with binding.pry!

Ruby Metaprogramming: Define_method and Instance_exec

This blog post runs through an example of iterating on some Ruby code. The bizarre fictional scenario develops to make the requirements ever more complicated. There’s certainly many different ways to solve the underlying problem, but I wanted to show an example where define_method and instance_exec can be combined with class methods to make for some very powerful and concise code.

For each change in requirements, I’ll post the code in full for how I’ve handled the new scenario.

I will say upfront that these two metaprogramming methods should be used with caution. Powerful and concise code does not necessarily make for easy to understand/debug/maintain code.

The Curry Restaurant

So, there’s a curry restaurant that sells Vindaloo and Tarka Daal and there’s a customer that likes to order curry from the restaurant… bear with me :). Upon receiving their meal, the customer will respond with a comment if the spiceyness isn’t to their taste. Unfortunately the restuarant isn’t consistent with the spices it uses when cooking.

With the above code we can do things like:

1
2
3
spicey_customer = SpiceyCustomer.new
spicey_customer.order_tarka_daal
spicey_customer.order_vindaloo

If any of the meals are mild then the customer will respond with “That’s too mild!”.

A new greedy customer

Then along comes another customer, a greedy one. They also like to order curry and comment on the meal. However, rather than comment on the spiceyness, they like to comment on the size of the portion. Unfortunately the restuarant is also inconsistent with its porton sizes, but the greedy customer doesn’t care what type of curry it is, they just want it large!

Already we have a bit of duplication between the two customers but it’s not a massive problem at this point.

A new menu item - Jalfrezi

The restaurant is doing really well and introduces a new Jalfrezi curry to the menu. The greedy customer responds the same to it as for any curry, but the spicey customer responds to the Jalfrezi, the same as when recieving a Vindaloo - they want it hot!

We’ve added order_jalfrezi to both customers and added expect_hot_curry to SpiceyCustomer because they respond the same for jalfrezi and vindaloo. Now we really do have duplication between the two customers, so we’ll look at refactoring that next.

Refactor - Dining Out

We decide that the two customers have enough shared behaviour to introduce a new module called DiningOut. This means both customers can order all three currys and then respond in a way that is specific to them. With the custom response defined in each Customer in the respond_to_meal method.

You might be thinking it would be better to make Customer a class and have GreedyCustomer and SpiceyCustomer as subclasses that inherit from it, but for this example it doesn’t matter. This is a version of the Template method pattern.

We have our new DiningOut module and we’ve defined a respond_to_meal in both customers. This code is definitely cleaner but there is still a bit of a smell.

If the restaurant expands with lots of new items on the menu and these customers need to respond differently, then the respond_to_meal method is going to get pretty big and complicated pretty quickly. We’re also very reliant on the name of the meal not changing.

In addition the DiningOut module will bloat quickly with each new menu item. That isn’t a massive problem, but there’s already a fair bit of obvious duplication going on where the name of the order method closely matches the argument sent to order_meal.

Now you might be thinking, why can’t we just expose order_meal as a public method and call SpiceyCustomer.new.order_meal(:jalfrezi) instead of SpiceyCustomer.new.order_jalfrezi. However…

A new fussy customer

Along comes the Fussy customer who will only eat tarka daal and the newest item on the menu, chips. The fussy customer isn’t interested in responding to the meal and just eats in silence. In addition, the Greedy customer will order chips, but the Spicey customer would never order chips. Sounds complicated doesn’t it?

The simplest thing to do is to add an order_chips method to DiningOut and an empty respond_to_meal to the new FussyCustomer.

Now we have some complicated rules and they’re not properly enforced. There’s nothing stopping us calling FussyCustomer.new.order_vindaloo or SpiceyCustomer.new.order_chips even though neither of those customers would order those things.

We could start putting some logic into the order_* methods to check the type of customer before ordering the meal, but that would get ugly fast. Ideally we want a solution where we can tell by looking at the code which customer likes to eat which food and each customer wont even know how to order food they don’t want to order.

Introducing define_method and instance_exec

Before we get into the final solution, the end result is that we can now do things like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
spicey_customer = SpiceyCustomer.new
spicey_customer.order_tarka_daal
spicey_customer.order_vindaloo
spicey_customer.order_jalfrezi
spicey_customer.order_chips # returns an undefined method error

greedy_customer = GreedyCustomer.new
greedy_customer.order_tarka_daal
greedy_customer.order_vindaloo
greedy_customer.order_jalfrezi

fussy_customer = FussyCustomer.new
fussy_customer.order_chips
fussy_customer.order_tarka_daal # returns an undefined method error

Also when we look at each customer we can see very clearly what they like to eat and how they like to respond for each menu item.

The nice thing is that if the restaurant adds a new menu item, we don’t need to touch the DiningOut module at all. We just need to decide which customers would like to order the new menu item and how they might optionally respond.

It works by defining a new class method eats. This method takes the name of the menu item and a block to be called with the meal so the customer can inspect the meal to determine their response.

The eats class method in DiningOut uses define_method to dynamically create the order_* instance method on the customer. This means if the customer declares eats :chips then they will respond to the method order_chips (and conversely if they don’t define it then they will return undefined method error). This makes it very obvious what menu items the customer will eat and prevents mistakes.

The order method uses instance_exec to execute the response block passed in. This is required because we want to ensure the block is called on the customer instance and not in the context of the class method - if we just did block.call(meal) then an undefined method (for whatever method the response block called) would be raised.

By accepting a block and calling it with instance_exec gives the nice benefit of that block being able to call any method it likes on the instance. In this scenario each Customer can call a different response method, for each type of menu item, if they want. This is important because, for example, the Chips menu item does not respond to the method spiceyness - so if there was a single assumed respond_to_meal method each Customer implemented it would have to handle every type of meal (checking if it responds to methods such as spiceyness and all sorts of other logic).

Conclusion

Obviously this is a ridiculously contrived and long example to demonstrate two metaprogramming methods in Ruby but hopefully if you’ve read this far, you’ve got something out of it.

As I mentioned in the intro, these metaprogramming methods should be used with caution. Powerful and concise code does not necessarily make for easy to understand/debug/maintain code.

Using Enzyme to Test a React Component onClick With Event preventDefault

Enzyme is a React Testing utility written by the developers at Airbnb that is designed to make testing your components much simpler and easier.

Here’s a quick tip for testing onClick functions where your code calls event.preventDefault()

When triggering a click via the Enzyme api method simulate, you are not actually triggering a real event – the underlying implementation is simply calling the onClick prop of the node.

So if you’re calling event.preventDefault() in your code, you will need to pass a second argument to simulate, which is a mock object, that is passed through to the event handler.

For example:

1
2
const wrapper = shallow(<YourComponent />);
wrapper.find('a').simulate('click', { preventDefault() {} });

Easy when you know how, but not immediately obvious. Fortunately the Enzyme docs are brilliant and well worth checking out for further detail and examples.

Making and Testing Ajax Requests With Axios in a Redux App

This article is going to explain how to perform and test Ajax requests in a React application using Redux and Axios which is a promise based HTTP client.

I like Axios because of the way you can centrally configure the common behaviour for the requests your front-end needs to make and the responses it receives. To take one example, it has the concept of interceptors that allow you to write a single function that can the handle responses such as a 401 unauthorized or 500 exception.

When using Axios I like to create a utility ‘Service’ class to setup the centralised config and ensure all Ajax requests get handled consistently.

It also means calls to Axios are not sprinkled all over the codebase, so I could swap out Axios for an alternative library and I should only need to update this one wrapper class.

In a Redux application, actions are purely functional and thus ‘side-effect free’. So in order to perform Ajax requests you will need middleware such as ‘redux-thunk’. An alternative, that I’ve not used but heard good things of, is redux-loop.

Redux Thunk means you can return a function from your action to delay the dispatch of it, or only dispatch if a condition is met (like the response status is 200).

Once Redux Thunk is installed, you can return a function in your Action as shown by saveResource in the example below:

Then for any containers that you want to use the action, you need to pass through the dispatch to the function in the mapDispatchToProps. See saveResource below:

To test the action you can use standard jasmine spy functions:

Then finally to test the Service class:

There are a few bits here to go through here. Firstly I found that when running the specs I needed to add the babel-polyfill for Promises. I’m using Karma test runner so I needed to add 'node_modules/babel-polyfill/dist/polyfill.js' to the files array in the karma.config.js file.

Secondly at the top of the spec you’ll notice that jasmine-ajax is imported. This is the official plugin for faking Ajax requests in your Jasmine specs. What I like about this is that it has no knowledge of Axios. In the beforeEach it is turned on and then all Ajax requests triggered in the specs are captured and can be inspected.

Due to the asynchronous nature of the promise, I use setTimeout to grab the most recent request and check it was formed correctly. To test the callback I use the then function on the promise under test.

A small, but important, detail is the call to done() – this ensures the test does not exit early and waits until the asynchronous part of the test is complete before exiting.

And that’s it! There’s a lot of code there, but once the Service class is setup it should remain pretty much unchanged. Adding additional Ajax requests to your React application actions should be fairly straight forward.

Using Capybara and PhantomJS to Test an External Website

Capybara is a great way for performing integration testing of web applications. Typically it is used in development with the default Rack Test Driver. However, the Rack Test driver does not support JavaScript and is unable to access HTTP resources outside of your Rack application.

Enter PhantomJS – a headless implementation of Webkit that can execute JavaScript, external urls and works with Capybara. The easiest way to get the two to talk together is via the Poltergeist gem.

I created an example repo with all the required config and setup via Docker: https://github.com/paulsturgess/capybara_phantomjs

But essentially it boils down to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
require 'capybara'
require 'capybara/dsl'
require 'capybara/poltergeist'

$stdout.sync = true

class Demo
  include Capybara::DSL

  def self.run
    new.run
  end

  def initialize
    setup_poltergeist
  end

  def run
    visit('https://google.com')
    save_page
  end

  private

  def setup_poltergeist
    Capybara.register_driver :poltergeist do |app|
      Capybara::Poltergeist::Driver.new(app, polyergeist_options)
    end
    Capybara.default_driver = :poltergeist
  end

  def polyergeist_options
    { js_errors: false }
  end
end

Demo.run