The Power of Toolkits

Disclaimer

This is not meant to be an article about why toolkits are awesome and super powerful; my goal is not to berate everyone who chooses not to use them. Instead I hope to provide some insight and context, based on my experience with toolkits, that allows you to decide for yourself whether or not toolkits might make your life easier, based on your specific product, team and requirements.

I should also mention that in this article I specifically look at toolkits for Javascript. While the general idea is hopefully easy to transfer to other languages and stacks, some of the things mentioned are very specific to the rough edges in the JS ecosystem.

TL;DR: in modern times, software development, especially in Javascript, usually comes with a lot of tools and setup. Bundlers, compilers, testing libraries, etc. All of these need to be configured. Toolkits can be very useful dealing with this across multiple projects. Third party toolkits come with trade offs and restrictions, so if the general concept works for you it might be worth considering creating your own toolkit to deal with your unique use cases and requirements.

What are toolkits?

Let’s start at the very beginning, to get everyone on the same page. What are toolkits?

For me, toolkits are libraries that help you build new projects without having to think too much about the underlying tools. Essentially, they expose commands that allow you to do the tasks you usually need to do (e.g. starting a dev server, creating a production bundle, running unit tests, etc.), allowing you to skip most of the tedious setup steps at the beginning of a project.

One famous example of such a toolkit in the JS world is react-script, which is used within CRA to deal with tooling like webpack, babel and jest — no manual config, setup or even knowledge about any of these tools needed. They’re all hidden behind the toolkit.

Toolkits are very popular, and easy to integrate. If you use CRA to create your app, your package.json probably contains something like this:

Why do we need them?

Without going into any details on how or why toolkits work, we can already tell they’re looks pretty neat. Anything that removes configs and complexity from our projects is always welcome 😄

But for clarity’s sake, let’s talk about why we need them. To answer that question, we must first ask: “why do we need tools in the first place?”

The fact that we need tools to create JS applications is actually a pretty controversial topic in the developer community. I won’t delve too deep here (you can read this blog post and many more for that), but people generally feel that as JS gets more and more complex, with more and more tools needed to build a simple “Hello World” page, the entry barrier for new developers is raised. Having to deal with tools and configs while you’re just learning to walk can be really intimidating.

But all of that is not necessarily true. Instead of using tools, we could just write all our Javascript in a single file, using syntax that all browsers understand and then manually add a reference to that file to our HTML. When I became a developer around 10 years ago, back in the jQuery days, this is how we did it. At the first company I worked for, we had no JS build pipelines and no extra tools were required to write production JS code. Here’s an overly simple (but hopefully somewhat relatable) example:

‍

1. Bundler

While this works, it’s not really maintainable or scalable. Plus it makes for a very bad developer experience. So, to improve things, we started to introduce bundlers.

Back in the day, bundlers included tools like grunt or gulp. Nowadays libraries like webpack, rollup or parcel are some of the popular choices, but there are so many that “picking the right one” can take a fair bit of time and energy.

What these tools will allow us to do is split up our code into a sensible structure of multiple files (called modules), making it much easier to find and scan the code parts that you care about, especially in large code bases. We can then simply import these modules wherever we need.

The bundler takes these and, well, bundles them back into a single file that we can then reference in our HTML.

Bundlers can also drive other DX improvements, like automatically refreshing the app whenever you change related code. A few years ago, this just meant triggering regular browser page reloads when changes were detected. Now, we’ve got advanced functionality like HMR and hot reloading (when it works 😉).

2. Compiler

As Javascript developers, we’re pretty spoiled. Our language is rapidly evolving, giving us new sets of improvements and new features pretty much every year. We refer to them as ES6, ES7, etc.

But the thing is, we can’t really use them to their full capability. Browsers will always take some time to actually implement these advancements. So, if we just started throwing them into our code, browsers wouldn’t have any clue what to do. They need some time to catch up.

To get around this, we started introducing compilers, which includes tools like babel, sucrase, swc, etc, allowing us to take advantage of things like arrow functions, promises or object literals.

The compiler’s job is to take the code that you write and transform it into something else — in our case something that browsers will understand. All you have to do is tell the compiler which target browsers and version ranges you actually care about.

3. Tests

We want to run unit tests in our code base, to ensure and increate the stability of our product. Again, there’s a variety of libraries to choose from for this job, such as jest, mocha or jasmine.

These tools also need to be configured — and, as we’ve already introduced bundling and compiling, we also need to make sure our test tool knows about these setups. Remember, you’re now writing code that needs to be compiled and bundled to make sense to other interpreters, be it other tools or the users browser. So not only do we have to deal with configuration for testing itself, we have to keep configurations of multiple tools in sync as well 😓

4. …

As I said, these are just 3 very basic examples. You might want to do more, like:

• specify how to deal with other file types being imported, e.g. images or SVGs
• set up to automatically do code splitting, to improve your app performance
• use babel and webpack to create multiple bundles, one for modern browsers and one for older ones, to reduce the amount of JS modern browsers need to download; or
• play with non-stable features that haven’t made it into the latest JS release yet and are still in one of the proposal stages, like decorators and optional chaining

I could go on listing more tools (e.g. storybook for better DX for creating new components, cypress or other tools for end-to-end testing, etc.) or specific features within the tools already mentioned, but I think I made my point clear. We can already see how tools and configs can start stacking up really quickly.

Still not convinced by the benefits toolkits can provide? Don’t worry — there is one more thing.

Bootstrapping

Another very useful and for whatever reason often overlooked feature of toolkits is bootstrapping. It helps you to get the basic file and folder structure set up in a single command, allowing you to create and start a new project within minutes.

This is the main thing CRA is does.

‍

The whole idea is pretty similar to what we used to do, before toolkits became popular: boilerplates. Copying files from a boilerplate into a new project or even just copying the last project’s code to create a new one. And there’s nothing fundamentally wrong with that. Boilerplates allow you to speed up the creation process of new projects and also ensures consistency between your projects — which is exactly what we want from the bootstrapping process of toolkits.

But there is a very important difference between the two approaches. Boilerplates contain all of the build setup and other tools. That means, the moment you copy the code from the boilerplate into your project, you make it that project’s responsibility to keep these tools and configs up to date and improving them over time. This can easily be a project in itself. You’re basically doubling the effort for creation and maintenance right from the start.

You also run the risk of creating “legacy code” faster. Each time you start a new project, using your learnings from the last project to change your tools and setups, you’re rendering past projects effectively dead (as least in regards to tooling).To stop this from happening, you’d need to pull all changes and improvements back into all projects manually … and I don’t know about you, but I’d never have the time for that.

This is another problem the creators of toolkits identified and tried to solve.

Dan Abramov explained it really well in his talk about the JS ecosystem and in the React blog post about creating apps without config. Toolkits make an attempt to split up these two concerns, tooling and config on the one side and boilerplates for the general file structure on the other. Tooling and config are the area most likely to change over time due to the evolving ecosystem, your business requirements and therefore the actual code are usually very project specific.

What you get in the end is the best of both worlds.

Why are toolkits so good?

Now that we’ve seen some examples why we need tools and toolkits, let’s look at some of the tangible benefits you might get out of them. Again, this is not an exclusive list, but some examples I’ve personally experienced include:

A centralised config

I’ve already mentioned one of the biggest benefits you might get out of toolkits: it allows you to move all tools and configs out of your individual projects into a single, easily maintainable source — usually in the form of a library.

Besides the decrease in maintenance cost, you benefit from an increase your ability to keep setups (and therefore your projects) up to date. Pulling changes and improvements back into older projects quickly becomes part of the natural process.

Enforcement of best practices

Toolkits can make it easier to enforce rules and conventions you want your team to follow and achieve consistency. Of course, you’re then able to make changes to these best practices more easily, e.g. through code mods. This, again, allows you to prevent codebases from older projects from becoming “legacy code” too quickly.

Enforcing these best practices automatically also helps new developers to get started in a codebase they’re not familiar with. Conventions can not only guide them through the code, but also help them write new code without the constantly worrying “am I doing the right thing…?”

Plus, it also saves developers from having to ensure consistency themselves. Win win!

Speed up new projects and prototypes

Toolkits can speed up the process of starting a new project immensely, by allowing you to use code in completely new ways, e.g. for prototyping ideas and creating quick proof of concepts.

This makes it easy to verify and iterate over ideas. It also m it super easy to turn a proof of concept into a real project, since you’re already using your production setup.

This extra speed and flexibility is great for your business, it means you can create new and exciting features more easily, making your product team happy. But the extra speed and flexibility, especially in the early stages of your initial product development cycle, can also help give you an edge over your competition.

What are the trade offs?

You can’t make an informed decision about whether toolkits are a good fit for your business without knowing all the facts. So, it’s important to also look at what implementing may cost you and what downsides might be attached to it.

Limited access to config

This is one of the more obvious trade offs, especially wth third party toolkits: you can’t always modify the configs for the underlying tools.

react-scripts, for example, doesn’t allow you to adjust the webpack config at all. This can be very limiting and frustrating when you’re dealing with unusual business requirements and edge cases. Instead, you’d have to “eject” your projects, which will copy all tools and configs from the toolkit into your app, which gives you full control, but also completely eliminates the benefits of the toolkit in the process.

However, that’s not an oversight or flaw in the toolkit. It’s very much by design.

You have to keep in mind that react-scripts is built for a very wide audience, all with different use cases and requirements. If everyone could extend the tools’ configs as they please, every time one of the tools changes, things would very likely start breaking. This, again, completely defeats the purpose of a toolkit, which is trying to to get rid of the need for you having to think about tooling and configs.

So, having limited access to config actually helps to decouple tooling and config from your project, which in turn helps enforce best practices (as it allows toolkit authors to help developers avoid common gotchas and problems and be more productive while building and maintaining their applications).

On a side note, this is one of the things I personally really like about CRA. All decisions they make in regards to supported tools and features, as well as default configs and available override options are usually very well communicated through public channels, e.g. via Github issues, twitter, etc. This not only shows the quality of the toolkit, but can also be very educational, even for experienced developers 😊

It also lessens the cognitive cost, as Dan again brilliantly shows in his talk “The Melting Pot of JavaScript”. He references a great blog article by Kathy Sierra, where she explains how too many options can affect your team negatively. Whenever you add something that requires a developer to make a decision, you’re using up bits of mental resources, which otherwise could have been applied to your project. It’s always important to keep this in mind before screaming at toolkits, demanding more flexible configs.

👎 Downside: very limited in regards to which tools and features you can use, depending on what the toolkit exposes

👍 Upside: not having the options can save cognitive resources, as well as enforce best practices, allowing you to learn and benefit from other peoples experience and expertise

Delayed access to new features

As a consequence of the above point, toolkits can prevent you from being able to use certain tools and features when they come out. The toolkit obviously needs to be updated to use the latest version of the tool in question to include the feature you want, but even then you’re still at the mercy of the toolkit actually using or exposing the feature.

This can be especially frustrating if you use the toolkit for prototypes and concepts, where you want to experiment with new ideas and tech.

On the flip side, you don’t run the risk of destabilising your project with new features, which are rarely stable from the get go. After several stages of alpha and beta testing, popular toolkits usually either implement the feature or made a conscious decision not to, and (hopefully) communicate the reason for that decision.

👎 Downside: makes it hard to easily experiment with new or unstable features

👍 Upside: enhances overall stability of your projects by not introducing unstable functionalities

Black magic

Generally speaking, the moment you hide something away and abstract it out, you create a layer of “black magic”. People don’t know how exactly that something works, all they know is that there’s something that somehow helps them make it work.

Toolkits hide tooling from your developers, which means they don’t have to think about what’s going on under the hood. This is, by definition, exactly what we want from toolkits, right? However, at the same time, it does mean they’re likely to not fully understand those tools. I think it’s important to be aware of the consequences a decision like using a toolkit can have.

Basically, hiding everything away is fine as long as it works. Developers who haven’t had to deal with tooling because the toolkit took care of it may not stay up to date with latest trends and ecosystems, making debugging and fixing issues a lot harder.

👎 Downside: can create layers of “black magic”, which becomes problematic when things don’t work

👍 Upside: when things do work, it allows you to build projects without having to think about tooling

So, are they worth it?

Hopefully, this article has given you enough information to make your own informed decision about toolkits and their usefulness.

Personally, I am a huge fan of toolkits. The amount of time saved building and maintaining projects can completely shift the way you approach product development, if you can align it with other processes in your team and company. If used right, toolkits can be an incredibly powerful part of your internal ecosystem.

At the same time, I’m a big advocate of creating your own toolkit if your requirements justify it. By that, I mean you tried available third party ones and have identified what’s missing from them to make it work for your specific use case.

Sure, this comes with an extra cost for developing and maintaining the toolkit, but I’d argue that cost is actually not as big as one might think. Building your own toolkit allows you to pick tools and setups that specifically match your projects and teams requirements, helping to enforce the conventions and best practices you want. And, once you have your own toolkit, dealing with a lot of the trade offs and issues mentioned above becomes much easier.

But my thoughts on creating your own toolkit are a whole other story for another article 😄

‍