A story about the beginning of PurpleStack

When I began working at Purple Technology, the number of developers could be counted on one hand. The choice of our technological stack was based on what the developer enjoyed doing that particular day and how well he could convince the rest of the team to use it (the two other guys). Usually the projects were more of a one-man-show, but for practically the same customers we serve today. In this article, I would like to try and describe how we moved forward as a team and how we choose technologies or architecture for projects now.

Punk times

As I already mentioned, the environment 4-5 years ago was straight-up start-up. We had a few projects for our clients – Client zone and Partner zone (Ember), one global API (CoffeeScript + HapiJS), forms (client registration, partner registration, account opening), and some websites. The only thing these projects had in common was JavaScript and making any kind of significant change in each part could only be done by one single person.

The technological race continued. The community slowly started to create a buzz around GraphQL-based APIs as an alternative to REST. At the same time, Filip joined our team, being only 18 years old at that time, claiming he is going to give the Serverless approach a go. The switching of the technologies was a wild ride, but at the same time, we gained a better insight into the JavaScript world and where it was headed. Being a small team, we began to spend hours discussing why we should go for a certain technology over another, why we should choose it now, and how long would it take to implement across all of our projects.

When the projects overgrows you

Now, imagine that the business is growing and you begin to feel pressure from the clients to improve the speed/quality/quantity of the product. That’s exactly what happened to us. Over the course of a few months we had to expand our team by several new people, and change work management so many times that Honza wrote a nostalgic article about the ordeal. We had to come up with a way to motivate the entire team and engage them in our key projects.

We were growing up, and suddenly it was the second half of 2019. Now we have a team of 16 amazing people: 1 product manager, 1 scrum-master, 1 tester, 1 team leader, 2 tech leads and the rest are developers. We are divided into smaller sub-teams, one of which I am in charge of. There are 5 of us, and we have a difficult task in front of us: integrate a brand-new system for onboarding clients into the client zone.

This means replacing the current registration forms with a new system, which allows the clients to make it as far as possible into the client zone, where they gradually fill in their information in order to unlock more functions. For example, a client will take a picture of their ID or passport to verify their identity, and unlock the option to deposit funds into their account online, automatically and instantly. Considering the upcoming plans to re-write the client zone, we began to discuss how to meet the requirements of the projects and avoid shutting the door for the future.

Onboarding vs. Client zone

The project we are talking about is a client zone, or actually its first part – client onboarding. It is one of the most important projects and a new version of it is already in development. When we introduce the client zone to our applicants at interviews, we explain it as something like this:

The client zone is actually something like internet banking – you can open various accounts in different currencies and on different trading platforms. You may deposit money, transfer them across your accounts, or withdraw it. There will be historical and statistical data with graphs in addition to that. Compared to internet banking, it is more simple as we do not deal with any mortgages, loans, or insurance.

Onboarding (the business term is Smart Onboarding System) was a relatively urgent project, which could not wait for the re-write of the client zone.

What to imagine under this project? We had a task to prepare the development stack for the new client zone, implement the onboarding part, and integrate it into the current client zone (in Ember. For simplification, we will refer to it as legacy zone from now on.

Onboarding consists of 3 parts. After registration, the client gains access to the zone and the usage of demo accounts. In the next step, the client can open a limited live account and finally a full trading account, at which point we require verification of client’s ID and a webcam selfie, filling of an investment questionnaire, or an upload of proof of residence.

Tech-stack, PurpleStack

We needed to build a stack which wouldn’t be dependent on individuals, and which wouldn’t lead to us being stuck with it after a while. We needed a stack based on microservices which will be flexible and robust. We needed a stack which would develop with us so that the whole current and future team can build a relationship with it. The three basic pillars we wanted to build upon were: 1. Serverless, 2. Infrastructure-as-code, 3. CI/CD.

Serverless could be described as an ecosystem of managed services with already defined design patterns, which allow the entire system to scale based on load. With that, new tools or frameworks come. From a business point of view, pricing politics is also interesting, as it’s often based on pay-per-request.

Filip is currently writing a series about Serverless – check it out; it might change your life.

The second point talks about infrastructure being defined in some descriptive language. It is the subject to the same development process as any other code and is a part of a versioned repository.

Both of these parts are covered by Serverless framework for us, where the infrastructure is defined in YAML or JSON format, together with all serverless services.

This infrastructure is further used in CI/CD, where a new instance of any version of the code based on a defined infrastructure can always be created, tests run on it, or the whole instance placed in the Cloud (AWS in our case).

When we were at the AWS conference a year later and saw this slide, I was so happy, because it had been a while since we started down this route, and yet we keep walking – or flying? 😊

Back to reality. We had a difficult task in front of us – to convince people from the business that it makes sense to invest extra time into this, to prepare it in newer technologies and have it useable in both the legacy zone as well as the new client zone. Our arguments worked: AWS will bring us lower expenses, more reliability, and scalability. For example, using AWS Step Functions automatically gave us product documentation for automated processes with a visual graph representation. It was important to emphasize that we want to build something which we will not use as a base for just one project, but something that will set the course for the future for the entire team.

The preparation of the technological stack took us several weeks and we are still developing it. You can read more about what our architectural solution looks like, and what kind of dilemma we had in a duel-article about our PurpleStack (as we now call it). The first part will be about the structure of the repository, about the reasons for choosing a monorepo, and how we dug deeper into the problems of Serverless and AWS, and which technologies or AWS services we used in the backend. The second part will deal with the Frontend and API needed to create a whole package. The whole time, we try to look for systematic and robust solutions and avoid any kind of shortcuts, which would bring non-standard or project-specific mechanisms.

When the future meets the past

We have managed to do what we set out to do; we developed a new Onboarding system as a separately encapsulated project without any ties to our legacy systems. It was possible to test it separately, and we could use parts of it as components. We have also included the integration into the legacy system at the lowest possible cost.

The next step was to integrate our developed onboarding system into the legacy zone. The registration section was separate from the zone, with the only commonality being the domain. Onboarding runs on its own url, but it needs an authorization token from the legacy zone or a Cognito authorization token from the new zone. The registration state automatically contained several steps which were supposed to apply changes in legacy databases or systems.

Onboarding in the zone appeared in different places. It was always an iFrame, in which a page from the onboarding app appeared. The authorization token is shared from the main app into the iFrame, and is then forwarded to the backend which returns the current step of the user.

In the legacy API, packages we developed for onboarding are used. It’s mainly about functionality; something which gets the user status and allows certain functionality based on that. It is used in the API as well as the FE. Besides that, there were three other parts which basically had to be duplicated in the legacy API in order to be used for the new onboarding system. These necessary adjustments in the legacy API, such as the iframe management and authorization of the user – which is about 1% of code which – will be discarded during re-write of the zone.

We did it, we are thrilled

What have we learned from this project? Architecture is extremely important. The requirements of the project were adjusted on the fly several times, and even today we have no idea what functionality the future versions of the client zone will have. For us, the most significant mark of success was our ability to create a technological stack based on microservices. We are vendor-locked with AWS, but our architecture contains a minimum of custom-made solutions, and it is so robust and flexible that it can react to a quickly changing fin-tech business. We have also learned that even seemingly simple projects such as onboarding of client can become a huge project, in which the architecture was the most difficult part.

The benefits we gained from the time invested are significant. From the design of the architecture to the client zone, the Purple Stack was created. We have used it in several other projects and in the future, we would love to offer it as an open-source template and provide a tutorial for using it. Another advantage is that, thanks to the microservices and minimum amount of custom solutions we passed the zone project with the complete onboarding to another team, and they seamlessly continue the development of the new zone. More about that next time.