One repo or separate repos?

Question

My organization is starting up an automation team, and we are still working on our structures and procedures.

We are using Git to store automation code, and we seem to be of two minds; some feel we should put all our automation code in one repo, and others feel we should separate code into multiple repos.

Here's the wrinkles: Much of the automation we will write is not for AUTs. It is for mission-critical third-party software, the actions of which may induce certain internally designed processes to do certain things. Most of these are integration layers between diverse third party applications. The overall software ecosystem is very large with diverse and mostly third-party applications. As a result, there won't be hardly any shared code between the frameworks for each application.

So far the best argument for single-repository design is that we can run all tests at once from one repo. There is no scenario, though, where we would run all tests at once.

I am proposing separate repos for each application we will automate, with an additional "common" repository for whatever few things we can share.

Also, there will be 6 automation developers, most of the time each will be the only one working on a particular application, for the small ones; but for the larger applications, there will likely be two or more working on it.

Do you agree with my camp of having separate repos? Can you give me pros and cons?

Answer 1

Running all tests at once is something that a Continuous Integration server should do for you, not something you do manual as it lets you wait for a long time. Let a server do the work and reporting.

Multiple repo's have the following advantages:

• Easier to schedule parallel-runs in a CI. Will be faster. (technically also possible with a single repo, but harder)
• Smaller, thus faster to work in with most IDE's (because they scan all the files often)
• Possible to version against the common library repo. This needs some more explaining. With a single repository you need to update all the applications if you update the common shared code. If you for example have an application tested that doesn't need updating anymore ever you could use an older version of the shared code, but if you have one repository you need to update all the code to use the new changes to the common shared code. (I think this one might be important for you, as it leads to less maintenance in the long run)

Cons:

• Testers might need to keep tens or more repo's update all the time.
• Harder and more time-consuming to set-up, although you could build a checkout script that updates all repo's at the same time. As if it was a single structure. You can also create submodules or subtree's with git as if it was a single repo, maybe worth investigating.
• Added complexity of a shared code library and versioning.
• One repo feels more like working with files in a single folder with sub-folders. This meets the non-god coders mental model better.
• YAGNI: you can always separate the code later when you have a real proven need for it.

Neutral:

• As git is very good with branches, multiple testers shouldn't be in each others way to much, aslong you merge daily with master.

My opinion:

Personally I would favour a repo for each application, as I like to version my tests with the application. Keeping the code-base simple for this application and I would know I have only the code I need, making it easier to clean and remove code that has become obsolete.

But I always advocate the YAGNI approach, which suggest to start with the simplest thing that might work first, because it might be good enough. Only add complexity if you really need it.

Conclusion:

So there is not a single good answer. I would try one or the other and evaluate regularly and adapt as necessary. It is not if one blocks the other. You can always split or combine in a later stage, just be open for this change and the slightly added work it will entail.

Answer 2

It depends.

Right now you might not see much need for a single repository design. Later, you may find your opinion changes.

You might want to use a single repository if

• you need to keep your different projects in sync. It's more difficult to keep separate repositories in sync than a single repository.
• your team members will need to switch between projects, particularly if they are having to regularly switch. The cost of switching project is significant enough without adding the need to make sure the correct source bindings follow.
• you need to be able to view quickly retrieve the a branch of all the projects. This is much easier when you have one repository than when you need to get latest for multiple repositories.
• you need to keep your branches in sync. It's a lot more complex maintaining a consistent branching policy when you're managing multiple repositories.

You might want to use multiple repositories if

• you don't and will not ever need to re-use code from one repository in a different repository.
• each team member is going to spend most of their time with one repository
• you have no need to use a centralized continuous build/continuous deployment process (and you don't foresee one) OR you are able to create a separate continuous build/deployment environment for each project.
• your separate projects use incompatible languages/development environments/life cycles.
• you can completely sandbox and isolate each project.

What it will probably come down to is which strategy most of your team members can work with. It is probably easier to start with a single repository and split out new repositories at need than to move in the other direction, but there really aren't many significant issues in either direction. All the factors I've listed are primarily conveniences.

Answer 3

Adding to the awesome and very detailed existing answers. There is a well-known success story of using a single large repository - Facebook and Google. As of 2014, Facebook's main repository was 54 GB in size. As of 2016, Google's main repository had 35 millions commits.

One of the most important advantages of having a single repo is atomic commits across multiple projects. If a feature consists of changes into multiple projects, you can do it via a single commit.

Some other advantages:

• single source of truth
• unified versioning and dependency management
• greater collaboration between teams, code sharing and reuse

Both of these companies are, of course, special cases because of the scale, but we may learn some lessons from how they organize, maintain and store code.

Follow-up read about having a single large repository:

• Monorepos in Git
• Google Is 2 Billion Lines of Code—And It’s All in One Place
• Advantages of monolithic version control
• Scaling Mercurial at Facebook (they did a lot of improvements to mercurial over the years)

Answer 4

I would apply KISS and YAGNI principle: Keep It Simple, and You Ain't Gonna Need It.

Start with one, and split into separate repositories when you can see the business need for it, and benefits.

I agree with @KatePaulk with pros and cons, but when no obvious benefits, I would go with simpler solution: single repository.