Each microservice normally is hosted in its own git repository, since it has very defined business boundaries and the code must be isolated from other microservices to ensure decoupling and deploy independance.

It may work greatly if you organize a team by each microservice. So, if a team is responsible for a given microservice and won’t work on other microservices, this organization may be good enough.

During project developments, we at Plataformatec understood that it is not so productive to focus on specific parts of a feature. Instead, we design and develop features by perceiving it as a whole, as it would be perceived by the end user. We don’t work with application specialists, we work with generalists and a lot of communication through pull requests.

So the best fit for the way we work, as our experience has shown us, is to put all the microservices and the clients that consume them into a single git repository. It may sound weird or semantically wrong for some, but after all, those microservices are small parts of a whole, something bigger, that is called software ecosystem. Since they share or exchange information among them, they’re somehow connected to each other.

This pragmatic approach is not exclusively ours, many people out there apply it. Two very nice examples are Facebook and Google. Of course their codebase is far larger than a normal application. They’re an exception. Google’s codebase, for instance, keeps really low level information like operating system configurations.

Using a single repository has proven to be a very good practice for us, because we can keep track of relevant pull requests easier; we can refactor, create and test new features throughout all the microservices faster; and test its integration without leaving the current context. Also, project gardening is way simpler: upgrading Ruby, Rails version, gem updates, using shared paths as gems, tests and deploy all of them can be automated and run across all microservices.

Have you worked with a single or multiple repositories? Please share your thoughts about it in the comments below!

The post Organizing microservices in a single git repository first appeared on Plataformatec Blog.

Just git clone the repo and you’re set!!

However, what we want with mirroring is to replicate the state of an origin repository (or upstream repository). By state, we mean all the branches (including master) and all the tags as well.

You’ll need to do this when migrating your upstream repository to a new “home”, like when switching services like GitHub.

As with most tools, there’s a lot of ways to accomplish that, but I’ll be focusing on two of them. The difference lays on whether you already have a working copy of that repository or not.

Mirroring a git repository without a local copy

If you haven’t cloned the repository before, you can mirror it to a new home by

$ git clone --mirror git@example.com/upstream-repository.git
$ cd upstream-repository.git
$ git push --mirror git@example.com/new-location.git

This will get all the branches and tags that are available in the upstream repository and will replicate those into the new location.

Warning

Don’t use git push --mirror in repositories that weren’t cloned by --mirror as well. It’ll overwrite the remote repository with your local references (and your local branches). This is not what we want. Read the next section to discover what to do in these cases.

Also git clone --mirror is prefered over git clone --bare because the former also clones git notes and some other attributes.

Mirroring a git repository if you already have a local working copy

By working copy, we mean a “normal” repository, in which you have the files that are being tracked into git and where you perform commands like git add and so on.

In this case, you may have a lot of local branches and tags that you don’t want to copy to the new location. But you do have references to remote branches. You can view them with git branches -r. If you pay attention to that list, tough, you may notice that you have a lot of branches that were already deleted in the upstream repository. Why?

Cleaning old references to remote branches

By default, when you do a git fetch or git pull, git will not delete the references to branches that were deleted in the upstream repository (you may view them in your .git/refs/remotes dir). We need to clean those old references before mirroring them to a new location.

To do so, run

$ git fetch --prune

This will update your references to the origin repository and also clean the stale branches reported by git branch -r.

Finally, mirroring the repository to a new location

Now we’re ready to send those updated references back to the origin repository:

$ git push --prune git@example.com:/new-location.git +refs/remotes/origin/*:refs/heads/* +refs/tags/*:refs/tags/*

Ok, what just happened here?!

We want those references inside the .git/refs/remotes/origin to be the LOCAL references in the new location. The local references there will be stored in the refs/heads dir. Same thing happens to tags.

The + sign indicates that we want to overwrite any reference there may already exist.

--prune means we want to delete any reference that may exist there if we don’t have such reference in our refs/remotes/origin/* (and tags) references.

Conclusion

Git is certainly not an easy tool to learn. Although when you do, it turns into a very powerful and flexible tool.

If you want to learn more about it, please see the excelent book written by Scott Chacon and available for free.

What about you? Have any tips on git you want to share?

The post How to properly mirror a git repository first appeared on Plataformatec Blog.

In the last months we’ve been working on a project with a mixed development team (Plataformatec’s team and the customer’s team). We, of course, used a version control system (specifically git) and we set up a nice git branching model for our team. As agilists, we know that we should not use anything that requires a lot of bureaucracy (things like opening a ticket to integrate a branch into the trunk).

Using nvie guide as base, we developed a git workflow. First of all, we had three main branches:

• production: contains the code that is currently on production. We also have a production server that contains, obviously, the production code up and running.
• staging: contains the code that is being tested before going to production (we used this branch to deploy to a production-like environment that worked as a final test until production, this environment is also called staging)
• master: contains the already accepted features. To consider a feature as “accepted” we deployed to another environment (called “dev”) and asked a QA analyst to test it. Once approved we merged the commits. This “dev” environment is used for this kind of approval and also for general purposes like when we say: “take a look at this new awesome feature we’re developing”.

For each feature that we developed, we created a git branch (almost all of them we pushed it to the remote server to facilitate code review and to deploy to the “dev” environment). Everyday, we ran git rebase master, to update our branch code (except for features developed by more than one developer). Once the feature is complete, we rebased master into it, and merge it using –no-ff (to create a merge commit). For the branches that more than one developer worked on, we usually talked and set up a “rebase period” where one does the rebase, forces the push (because you changed your local tree so git does not accept it as a non forced push) and updates the remote branch.

Close to the production deploy, we merged (always using –no-ff) master branch into staging branch and deploy to staging. Once approved, we merged to production, and created a tag telling the current version of the application and then we do the deploy. When we deployed to production we also removed the merged branches from the remote repository.

One of the great advantages of this schema is: Master is always “almost” ready for a release. Yeah, some features really deserves to be validated right before the deploy, because another feature can break them, but we kept the master as an “always releasable stable” branch (and also we used a continuous integration tool in order to enforce all tests passing). Another great advantage is: as we updated our code everyday, it was very unusual for us to face big integration scenarios.

For the “dev” environment deploy we also set up a capistrano task that asks which branch we want to deploy to be possible to deploy something from any branch.

This workflow has worked really well for us and maybe it is useful to you (maybe for you to adapt it to something that works better for you as we did with the workflow suggested on nvie).

To summarize, this is our git workflow in commands (supposing that we are on master branch):

git checkout -b my-awesome-feature

(... you do some code and some commits and you go home to have some sleep or maybe play some starcraft 2 ...)

#(arrived at the office on the next day)
git rebase master

#(... continue working and commiting and sleeping (or maybe playing some starcraft 2 ...)
git rebase master

# (... some commits ... and voila ... you've finished...)
git push origin my-awesome-feature
cap dev deploy


# (...YAY! QA analyst just approved it ...)
git pull origin master
git rebase master


# (run the tests to ensure all of them pass)
git checkout master
git pull --rebase origin master
git merge --no-ff my-awesome-branch
git push origin master

#(... it is time to validate on staging)
git checkout staging
git pull origin staging
git merge --no-ff master
git push origin staging
cap staging deploy

#(... QA analysts validate the staging ...)
git checkout production
git merge --no-ff staging
git tag -a v1.4.2 -m "Releasing on 13th February"
git push origin production
git push --tags origin production
cap production deploy


Well, this is how we improved a git workflow based in another one. As almost everything, there is no bullet proof for it, but we found interesting to share this experience with you as it was a success (every developer on the team enjoyed it). But please, we would like to receive some feedback about it :-). Have you used something similar in your team? Do you have any ideas on how we can improve it?

The post A (successful) git branching model first appeared on Plataformatec Blog.