3 decisions that shaped the Polaris UI

Imply Polaris is a fully managed database-as-a-service for building realtime analytics applications. John is the tech lead for the Polaris UI, known internally as the Unified App.

It began with a profound question: What if we took every UI from our entire product suite and smooshed them together? And thus, the Unified App (a.k.a. Polaris) was born.

Historically, our product lines were divided across team boundaries and those divisions applied to everything, including source control, packaging, deployment, and overall user experience. Bridging those gaps at the UX level required a different high level approach to UI engineering. Rather than horizontally aligned teams building standalone products in isolation from each other, we were going to be vertically aligned teams building a single product collaboratively.

How do we organize the code? How do we deploy it? How do we share libraries and state, both between individual Polaris features but also between Polaris and Imply Enterprise? Answering these questions resulted in 3 key decisions that shaped our new approach to UI development.

Decision 1: Put it in a monorepo

Monorepos: so hot right now.

Sooner or later, it makes sense for any sufficiently complicated frontend project to structure itself as a monorepo (in my humble opinion). In fact, I would argue that there are only two reasons not to develop in a single repository. 1) Different packages have different visibility, e.g., some are open source and some are proprietary¹. 2) There is so much code and there are so many contributors that it becomes impractical to manage from a technical or organizational perspective without advanced in-house tooling.

For every other case, I can’t imagine ever being as productive with multiple single-package repositories as we are with a monorepo.

These are just some of the benefits we get:

• Can test changes to internal dependencies immediately, without “npm link” or tools built to address the many shortcomings of “npm link”
• Can make breaking changes in a dependency and update all usage sites in a single commit
• Updates to shared 3rd party dependencies are synchronized and can be done in one shot
• All in-flight UI work is funneled through a single Pull Requests page

That last point is especially powerful because it means there is virtually no barrier to cross-team code reviews, which helps us maintain and teach standards at the org level.

Our monorepo uses npm workspaces, which have been available since npm v7. Other tools like yarn and pnpm may have some technical advantages over npm, but it’s hard to beat The Default when it comes to onboarding new engineers or building CI pipelines. Nearly all of our packages are used exclusively within the repo, so we don’t need Lerna² or other similar publishing tools.

Any choice like this is a tradeoff with potential downsides. For instance, our UI monorepo is private, but we have a handful of open source dependencies that can’t be assimilated, so they must be kept separate. I consider this a problem because our monorepo workflow is so productive that it makes the “old way” especially painful and potentially error-prone.

A more pressing issue, however, is that the rate of growth for our UI projects has increased substantially, which has downstream consequences on things like IDE responsiveness, CI/CD pipeline execution time, and overall DX. We’re not at a breaking point yet, but we certainly feel the weight.
The solution to this will be multi-faceted. We will likely adopt Turborepo for caching build results and only executing build steps for dependencies that have actually changed. It’s also very likely that we will adopt esbuild as our bundler to significantly reduce artifact build time. Both of these changes will be detailed in follow-up posts, whether or not they are successful.

¹ In which case, maybe you just need 2 monorepos.

²  But we still use it for now as a change-aware script runner

Decision 2: Ship a monolith

Spaces vs. Tabs – Emacs vs. Vim – Monoliths vs Micro Frontends

At the start of this project, we surveyed the web architecture landscape in search of novel approaches for building complex SPAs. One intriguing idea that started gaining popularity in 2017 was micro frontends, or, loosely coupled UI fragments that are built independently from each other but are composed into a single user-facing application. One way to build micro frontends is with a Webpack feature called Module Federation – there are other ways, but this seemed the most suitable for our stack.

Going in depth on these ideas is beyond the scope of this post, but if you want more information, micro frontends have a website, a book, and lots and lots of articles. Module Federation has some high level documentation and an article and a repo full of examples.

As interesting as it seemed, we decided against micro frontends for Polaris.

Why not micro frontends? On the surface, it seemed to align nicely with our desired org structure, with teams focused on delivering vertical features. Each team could have their own distinct frontend app that gets stitched together at runtime to compose the full application. Teams could operate somewhat independently from each other and focus directly on their slice of functionality.

One problem is that our organizational aspirations did not yet reflect reality. We identified several functional teams, but didn’t have enough frontend engineers to actually staff them. This meant that, in effect, one small team was building features across all functional areas of the application; any software-level divisions we created to reflect the org structure would have been completely artificial.

Another problem is that some of the new functional teams had cross-cutting, foundational purviews. For instance, the Identity and Unified App teams did not fit neatly into this vertical-feature world and would end up contributing to many micro projects, which kind of defeated the purpose.

At this point, it seemed that micro frontends were a mismatch. But if our growth targets became reality, it would be worth at least looking into Module Federation at a mechanical level, so I did.

I didn’t get it.

I mean that literally – I did not fully understand the mechanics of Module Federation, no matter how much I read. The official documentation was simultaneously too high-level and too low-level, if that’s possible. Yes, there were example projects and blog posts by the people who contributed the feature to Webpack, but I found it all to be rather impenetrable. Had we been really sold on the approach at a high level, it might have been worth investing more into research and experimentation, but this smacked of a solution in search of a problem.

Instead, we built a monolith.³ Polaris’s high level features are exposed as React components that are imported dynamically and lazy-loaded by a single entry point. This entry point is bundled by Webpack into a few JS chunks and deployed to S3. All features are part of the same project and have received contributions from multiple teams, even though each feature might officially have a single team “owner”.

It works.

Many of the advantages and disadvantages of monorepos I listed above also apply to building a monolithic application, but I maintain that the advantages prevail. Cross-team collaboration has no technical barrier because we’re all using the same tech; new patterns we discover can be rolled out across the entire application at once; and we can run the entire application locally using a single command.

There may come a time when we hit the limits of a monolithic architecture, but that will not be soon. If and when that time comes, Imply will be a very different company at a structural level. And maybe – hopefully – the documentation for Module Federation will have improved…

³ Technically, a few monoliths: Polaris UI, Polaris Admin, Polaris Sign-Up, Imply Enterprise, and Clarity.

Decision 3: No to Redux, yes to global state

You might not need Redux, but your app needs global state.

We may never agree on whether React is a library or a framework⁴, but we will probably agree that it’s not very useful on its own for non-trivial applications. When we choose React, we are choosing to choose a base component library, a styling system, a querying layer, and a global state container – plus a bunch of other things depending on what we’re building. Batteries not included. For state containers, the choice usually begins as a binary: Redux or Not Redux.

We chose Not Redux.

This decision was not because we opposed immutability or state/reducer patterns – we love those! Rather, the team had an historical aversion to Redux due to its dominating personality, i.e., it wants to be your one and only state container. Also, simply speaking, Redux Toolkit (RTK) was not as mature or widely used at the time we were making these choices. Had we started this project even 6-8 months later, there’s a good chance this section would be titled “Use Redux Toolkit”.

What we chose instead was a spectrum of state management techniques for different UI paradigms. Sitting in the default position is react-query. Though not a traditional state container, its own documentation challenges you to think of it as a system for server state management. Since results are cached and deduplicated, queries offer many of the benefits of traditional global state without the requisite boilerplate.

React-query is a good fit for us because Polaris is heavily biased towards reading state. If we were building a write-biased system, a.k.a. a glorified CRUD app, queries-as-state wouldn’t be enough. We would likely need Redux or MobX or even a completely different approach like Remix.

Not having a single global state tree also helps us maintain the illusion of isolation between our different application slices even though we’re shipping a monolith. Queries typically sit at the top level of an application slice and the results get passed into child components. Different sub-applications might use the same query-state as each other, but they don’t know or care. When one app fetches state that is used by another, the other app will get the cached value and appear to load instantly. Updates to that state will be reflected in both applications when the cache is invalidated.

All of this is possible with Redux, of course, but the key difference is that with queries-as-state, we don’t have to decide where that state will live in the tree. In other words, we made a single decision that helps us avoid making more decisions.

Of course, there are parts of Polaris that don’t fit neatly into the queries-as-state model, and for those we need something more traditional. A well placed “useReducer” exposed via Context can take you really far, but it falls apart in performance-sensitive views, such as with large data grids or complex visualizations. It’s not currently possible for individual components to subscribe to part of a Context value and ignore unrelated updates, which means it’s either very difficult or downright impossible to avoid performance-killing re-renders with this approach.

React-redux avoids this problem by moving subscriptions into userland, but we’re committed to not using Redux. Luckily, state containers are somewhat of a cottage industry in the React ecosystem – you might even call it an embarrassment of riches. One such container we have used with great success is Zustand.

Zustand is great because it’s a bit of a chameleon and doesn’t impose a strict ideology on its users. It’s also perfectly happy storing non-serializable state, which can be pretty awkward with Redux. And unlike newer state libraries like Recoil, Zustand can be used with class components too. Its default mode is global, but with provider components you can have parallel trees of the same state structure. And yes, it does userland subscriptions, which means it’s also suitable for performance critical functionality.

One of the most satisfying changes I made early on in Polaris was migrating the state of our ingestion application to Zustand. At the time, it was using the reducer+context approach but its performance wasn’t scaling well with the complexity of the UI. Thanks to Zustand’s redux middleware, I was able to port our reducer, state, and actions directly and with 0 line changes into a Zustand store. The difference was night and day from a performance perspective, but the architecture of the state tree itself was almost indistinguishable.

The combination of react-query, high level component state, and Zustand-when-needed has been a great decision for us so far. Though it might lack the uniformity of an all-in-one state solution like Redux/RTK, it works on a practical level and helps us distribute our state across sub-applications without worrying about conflicts or taxonomy.
Wow! You made it to the end of this post. Have you considered a career in frontend development?

⁴ It’s a framework. Fight me.