17 criteria to kickstart your research and evaluation of third-party libraries

Why reinvent the wheel? As developers, we often find ourselves reflecting on that question. Whenever we are about to introduce a new piece of functionality, we tend to look around to see if the existing tools, libraries and frameworks available out there could facilitate our work. After all, is it really worth investing time and effort to replicate something that might already have been created and optimised? Or would it be better to make use of the existing solution and adapt it to accomplish our goal more efficiently?

Most of the time, it all comes down to whether the existing solutions suit our technical needs and to what degree they align with our business requirements. For this reason, carefully researching and evaluating third party libraries has become a very common activity among developers. Unfortunately, researching and identifying the most appropriate third-party library is a complex task and comparing different libraries can often become overwhelming.

At Panaseer, we realised that establishing and making use of an evaluation framework comprising a set of development and business criteria can make the process smoother. A well-defined list of critical areas to investigate provides a clear direction during the research activity. On top of that, evaluating multiple libraries against the same requirements allows for more open and transparent discussions.

The following list is a result of a research in its own, through which we have identified a set of criteria that we use for making a decision on whether to adopt and integrate a third party library at Panaseer. While this list reflects the areas that we consider essential for our business and development stack, we hope it will encourage you to establish your own set of criteria that suits your vision and culture.

1. Purpose

What is the core functionality offered by the library?

Although different third-party libraries solve the same problem, they often have distinct purposes in mind and thus offer additional unique functionality. By noting this we can determine which one aligns the most with our requirements, present and future. For example, when evaluating visualisation libraries to generate bar charts, a specific library might be more suitable than others because of the additional chart types it provides.

2. Technology

What technology is the library based on? How well could it integrate with the existing codebase?

We want to consider each library’s technology due to the impact it might have on user experience and development work, including integration with the existing codebase. Back to our visualisation libraries example, a third-party library could render elements through SVG whereas a second one might do that using the Canvas API, which can influence both implementation activity and performance.

3. Development Kit

What does the library provide in terms of development kit? Is it likely to speed up or slow down development time?

Beside the underlying technology, we investigate the actual development kit each third-party library provides. For instance, we look at whether certain libraries come with useful tools or comprehensive APIs that can potentially facilitate the implementation of the required solution.

4. Documentation

Is the library’s documentation comprehensive? Does it include examples and code snippets? Is it easy to navigate?

A well-written, exhaustive and up-to-date documentation is aways valuable, even more so when developing a piece of functionality with a new third-party library. A simple browse through a library’s documentation is often sufficient for us to have a feel of what to expect and take that into account when making a final decision.

5. Cost

Is it a free or a premium library? What would be the approximate cost, if any? What are the available purchasing options?

Determining what are the monetary costs involved in adopting a library is very important for a business perspective and can ultimately make the difference in the decision making process. While it is relatively straightforward to compare premium library prices, we should equally consider the development time and the associated costs that each library would require to deliver the desired solution.

6. Releases

What is the current version of the library? Does the library receive regular updates? Does it follow a release strategy?

The release history of a third-party library is a good indicator of its release strategy, providing an insight on what can be expected for that library in terms of updates, bug fixes and major changes. These factors often play a major role when we decide what library to pick. A well-supported library with a reliable release strategy can often extend the lifespan of the features built with it.

7. Size

What is the actual file size of the library? How much will it add to the existing bundle size?

Although not necessarily crucial, it is still worth looking at the size of the library files, especially in their minified versions, and reflect on how much that impacts the existing bundle size. Usually, the smaller, the better.

8. Community Support

How large and active is the community around the library? How many developers have contributed to the library source code? How easy is it to find information and help online?

Depending on how popular or recent the library is, it can be more or less difficult to obtain information, tips and answers to questions online. Having an idea of what to expect in terms of free support can determine how suitable a third-party library really is. We usually do this by collecting simple metrics through quick online searches, such as number of library contributors on GitHub or search results for the library on StackOverflow.

9. Professional Support

Is there any professional support available to get help with the library? Is the support provided by the library creators or by external freelancers and consultants?

In cases when third-party libraries are not open source, it is generally more difficult to obtain information through online resources such as GitHub and StackOverflow. As such, we look for what are the available options to get support while working with the library, what professional services are provided and what are the costs involved.

10. License

Under what license is the library published and distributed? What are the associated requirements?

Open source libraries come with licenses that generally allow software to be freely used, modified and shared, while premium libraries very often include a precise list of strict requirements. Depending on its license, a third-party library can be more or less suitable for a certain company, application or piece of functionality. We therefore evaluate the license aspect before adopting a specific library in order to make sure we understand the corresponding obligations and avoid potential legal troubles in the future.

11. Performance

How performant is the library? Is the expected performance aligned with technical and user requirements?

Assessing a third-party library’s performance before integrating it into the existing codebase is probably one of the most challenging tasks. At a minimum, we try to find out information and metrics by researching online, either in the library’s documentation or, even better, through other developers’ experience. From our perspective, a library’s performance should at least meet the technical and user requirements identified for the proposed solution

12. Browser Compatibility

What browsers are supported by the library? Is there any critical browser missing?

Cross-browser compatibility often represents a pain point for front-end developers, even more so when making use of third-party libraries. In our experience finding out information about what browsers are supported by a given library should be fairly straightforward and certainly worth it.

13. Device Compatibility

What devices are supported by the library? What are the supported interaction modes?

Beside cross-browser compatibility, third-party front-end libraries often need to be evaluated against supported device types. For example, depending on the specific solution being built, it might be a requirement to select a library that fully integrates touch functionality for a specific mobile operating system. Even when not explicitly required, we feel it is still a good idea to list compatible devices when going through a shortlist of libraries, if nothing else to assess their potential for the future.

14. Data Format

What are the library’s data format requirements? Does the library align with a standardised data structure?

While all third-party libraries handle data of some sort through input and output, some of them can have strict requirements in terms of expected formats or structures. To us, listing and evaluating them makes it easier to understand whether a library can be easily integrated into the existing codebase from a data perspective.

15. Look & Feel

Does the library have a distinctive look and feel? How customisable is it? Can it be aligned with that of the existing application?

Whenever a third-party library includes a visual output, we believe that its appearance should not look too far from that of the existing application. For this reason we think it is a good idea to research how customisable a library’s look and feel is, what technology is the styling based on as well as what would be the time investment required to implement the desired appearance.

16. Extendibility

Is it possible to extend the library’s functionality? Is it encouraged? Are any third-party plugins or extensions already available?

Depending on the feature being built and the functionality provided out of the box by the library, it might become necessary to extend or optimise the library’s core components. Not all third-party libraries allow it, as this really comes down to the license under which they are distributed. To us, given the long-term impacts of adopting a specific library, understanding the legal and technical constraints around its extendibility is critical.

17. Future Direction

What is the vision for the library? Is a roadmap available? Does it align with that of the existing application?

Information about a library’s future plans should always be taken into account when evaluating third-party libraries. We believe that this not only provides an overview of upcoming features, improvements and bug fixes, but also prevents the adoption of any library that could soon stop receiving support or even be discounted entirely.

As we already mentioned, identifying and agreeing on a set of third-party libraries evaluation criteria can serve multiple purposes, from providing guidance for the research of information to serving as a common basis for discussion and decision making. Naturally, to fully serve these purposes the list itself should reflect both the nature of the libraries being evaluated as well as the business and technical requirements that have been set for the proposed feature. Criteria can therefore change depending on whether the evaluation revolves around front-end or back-end libraries, on existing time constraints and, of course, on lessons learned from previous evaluation processes.

Despite being a simple set of areas to investigate, at Panaseer the use of such framework has already proven very valuable for the adoption of new front-end libraries. Not only, as a team we are continuously assessing, extending and optimising our list of criteria. After all, the best part of this framework is that it can be easily adapted to fit each research purpose and context.

How about you? Do you have an established framework for evaluating third-party libraries? Do you feel like we are missing any relevant criteria? Let us know!