Skip to main content

Yaml Simple Copies from Git

How did it work before?​

Previously we had Simple Copies stored in Postgres DB and a caching layer in Redis.

how it worked before

This may work fine for some time, but when we have to do ~30 network calls to Redis (btw, Redis is hosted on a different server) to render a single page then it starts to sum up and increase server response time.

Overview of new approach​

New implementation uses Rails’ built-in I18n module which provides translations via bunch of  .yml files. These files are located in the root folder of the project and there's no need to do any network calls to read them. I exported all simple copies from https://www.powr.io/simple_copies.json, converted them into yaml files and pushed them into new repo https://gitlab.com/powr/copies

Here’re some advantages of new approach:​

  • much faster (no need to do Redis or Postgres calls)
  • no need for optimization code
  • copies are synchronized across all environments (production/staging/dev/review-apps)
  • no need to write migration files to add/update existing copies

But there’s a bit of a learning curve how to add/modify copies in Git Repo:

  • you have to push changes to git
  • search is not very convenient, but I made a tool that makes it easier

How Deployment Works​

  1. There's a new step in Gitlab-CI script: bash deployment/clone_copies.sh
  2. That command clones COPIES_REPO into /config/locales/copies folder and creates a file config/locales/last-commit-sha.txt with the latest commit SHA.
  3. Docker starts building an image with freshly cloned copies.
  4. During the app boot we check config/locales/last-commit-sha.txt content and latest commit from the COPIES_REPO
  5. If results differ then we run bash deployment/update_copies.sh which pulls new copies and updates config/locales/last-commit-sha.txt with the latest commit SHA
  6. Otherwise we do nothing (copies didn't change)

Development with new Yaml Simple Copies​

First of all, please add COPIES_REPO: https://gitlab-ci-token:[email protected]/powr/copies.git to your application.yml

Steps 4-6 from Deployment work for local development too.

New Syntax Alert​

Previously we used double curly {{braces}} for Ruby placeholders and single curly {braces} for JS placeholders. Now we will use %{this} for Ruby placeholders. It's a I18n built-in functionality for placeholders.

Also previously we had some invalid Yaml keys in the codebase. For example, we cannot have simple copies with keys general.usage_pricing.v_500.label and general.usage_pricing.v_500 in yaml files because general.usage_pricing.v_500.label is nested under general.usage_pricing.v_500 and that's why I had to replace . with _ for general.usage_pricing.v_500_label

Adding or modifying Simple Copies in Git Repo​

  • Clone repo https://gitlab.com/powr/copies
  • npm install
  • npm start will start small local app that allows you to search and modify simple copies across all Yaml Files.
  • node version has to be >= 14
  • After adding/modifying copies you need to commit and push your changes to master branch
  • There's a precommit hook that validates Yaml files. copies.json is modified too because it includes all copies across all yaml files for faster search, that's why it's also required to be committed and pushed
  • Demo of using the tool: https://www.loom.com/share/4e5794741f2942a98af13cec600f2a68

How does staging or production get new copies after pushing to the COPIES_REPO?​

Server needs to be restarted after COPIES_REPO is changed to get new copies. If you changed copies AFTER deployment you can manually restart your environment from heroku:

But most engineers don't have access to production app on heroku, that's why there's a GitLab-CI job that can be triggered to restart production app: https://gitlab.com/powr/copies/-/jobs

Please add your name to the doc if you finished reading it:​

Sergey, Abylay, Nazir, Anuarbek, Praneeta, Brent, Ben, Valeriy, Rustam, Amir, Aigul, Yevgeniy, Alikhan A.

FAQ​

How can I add a new simple copies to my branch?​

COPIES_REPO is the only source of truth for all simple copies. You just need to push your copies and restart your server. Please add/modify copies using only the Simple Copy Editor tool in the COPIES_REPO. Run npm start and open http://localhost:3023

How to use simple copies on the client side?​

This approach wasn't changed. If your copies are "app-specific", then you

  1. Create and push simple copies to COPIES_REPO2) Restart your local server to fetch new copies

  2. Add their KEYS to corresponding config/simple_copies/[apptype.downcase].yml

Example:​

you created and pushed following copies to the COPIES_REPO using Create multiple copies

productreviews.settings_tab: Settings
productreviews.design_tab: Design
productreviews.blabla_tab: Blabla
productreviews.hello_tab: Hello

Then you need to restart your server.

And now you only need productreviews.settings_tab and productreviews.design_tab on the client side. Create config/simple_copies/productreviews.yml and add those keys in YML format:

productreviews:
  settings_tab:
  design_tab:

There's a simple_copies_dictionary method in SimpleCopyHelper, it will load copies listed only in that file into client side's SIMPLE_COPIES_DICTIONARY