How to merge two Remote Config instances

Prerequisites

brew install jq 

curl -fsSL https://bun.sh/install | bash &&
touch ~/.zshrc && echo 'export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"' >> ~/.zshrc &&
source ~/.zshrc

Introduction

When managing two Firebase Remote Config instances (such as Production and Development) it’s common that the instance properties go out of sync. This could be a change to a common property in one, or when a new property is added to the other.

Imagine you’re managing two Firebase Remote Config instances: one for Production and one for Development. Here’s what each configuration looks like:

Production

{
  "featureA_enabled": true,
  "max_items_displayed": 10,
  "env": "prod"
}

Development

{
  "featureA_enabled": false,
  "max_items_displayed": 8,
  "env": "dev",
  "new_feature_enabled": true
}

In this scenario, the featureA_enabled key in the Production instance has been updated to true, reflecting a recent feature release, but the Development instance still holds the outdated value of false. Meanwhile, in the Development instance, a new key, new_feature_enabled, has been added during ongoing testing and development. The problem arises when trying to keep the values of shared keys (like featureA_enabled and max_items_displayed) in sync, while also ensuring that new keys (like new_feature_enabled) in the Development instance are preserved during the merge process. Having to manually sort this differences is error-prone and time-consuming.

A simple solution

Fortunately this can be solved easily using the jq command-line tool, which we can use to merge both JSON files into a single JSON file.

Keep in mind that the order of the parameters is important.

jq -s '.[0] * .[1]' development.json production.json > merged_file.json

Testing the solution

Merging the production and development JSON configurations above results in the following:

{
  "featureA_enabled": true,
  "max_items_displayed": 10,
  "env": "prod",
  "new_feature_enabled": true
}

Visualising file differences

To easily visualise the difference between the two files, we can use the diff command, which takes two files as inputs and it will output only the differences between them.

diff <(jq -S . ./production.json  ) <(jq -S . ./merged_file.json)

For example, running the diff tool over our previous example will give us the following output:

Note that max_items_displayed is also highlighted because, in the JSON scheme, the last item must not have a comma, and since we’re adding a new key below, it is marked as “changed” because of that trailing comma.

Solving Edge cases

So far, we have been able to merge the configuration of Production onto Development while keeping new keys added in Development, but it turns out that Remote Config stores conditions and rollouts within the configuration file, which means that if, in your Development instance, a developer was making changes to any of these non-labels. i.e. by adding a rollout or creating a new condition, those are going to be present on the production instance, which we definitively don’t want.
To solve this we can run a script that strips all of that unnecessary data from the Development instance file before merging Production into it.

For that, we will use Bun, a JavaScript runtime to create a script that looks for the file development.json , removes all unnecessary keys and outputs a development-modified.json
You can paste the following text into a file called prepare.ts

// prepare.ts
import fs from 'fs';

// Load and parse the JSON file
const data = JSON.parse(fs.readFileSync('./development.json', 'utf8'));

// Remove conditions section
delete data.conditions;

// Remove version section 
delete data.version;

// Process parameter groups to remove conditionalValues
for (const group of Object.values(data.parameterGroups)) {
  for (const param of Object.values(group.parameters)) {
    if (param.conditionalValues) {
      delete param.conditionalValues;
    }
  }
}

// Save modified JSON
fs.writeFileSync('./development-modified.json', JSON.stringify(data, null, 2));

And then run it:

bun prepare.ts

After creating our development-modified.json file that has no rollout and condition keys, we can use jq to merge the files and verify the difference, while ignoring those configuration fields from dev.

jq -s '.[0] * .[1]' development-modified.json production.json > merged_file.json

Then, we can verify the differences between our current production file and our merged file that does not contain extra non-labels keys on the file.

diff <(jq -S . ./production.json  ) <(jq -S . ./merged_file.json)

Summary

To merge two remote config instances, you need:

1. Download the FBRC current config file from both development and production instances

2. Save them in the same folder (`[Your Folder]`) and rename them to development.json and production.json, respectively

3. Open your terminal in the same folder (`cd [Your Folder]`)

4. (Optionally) Erase any non-label from the dev instance by executing the script given in the Solving Edge cases section

   1. Make sure to create the script in the same folder ([Your Folder])
   2. Then run the script

      bun prepare.ts 
      

5. Merge Production into Develop

   jq -s '.[0] * .[1]' development-modified.json production.json > merged_file.json
   

6. Visualise the difference

   diff <(jq -S . ./production.json  ) <(jq -S . ./merged_file.json)
   

7. If the files are the same, there shouldn’t be any text logged in the terminal