Managing dependencies

Dr Maeve Murphy Quinlan

What are dependencies?

Dependencies are the versions of various different modules, packages, or software that your research code depends on: all of your imports.

Ruby has a particular computing environment she has developed her code from. This computing environment is represented as a bubble above her computer with various hexagons with version numbers as well as Rstudio and R installed on her computer. Her code ran just fine on her particular computing environment. Avi attempted to run Ruby’s code on his very different local computing environment and got an error. His computer runs the same code but came up with a different result!

Don’t be like Ruby and Avi! Record your dependencies.

Image CC BY Candace Savonen, retrieved from Reproducibility in Cancer Informatics.

General Rules for Dependencies

Managing dependencies is something many, many people find complicated and difficult. Let’s start with some key ideas to simplify things.

NYU Libraries provide some straightforward guidance that is language agnostic:

  1. Use dependencies wisely and sparingly: don’t import dependencies you don’t actually use, and try to stick to libraries that have robust and stable releases.
  2. Use a package manager of some form: package management software, virtual environments and containers help to avoid conflicts and incompatibility errors between packages.
  3. Record your dependencies in a metadata file: package managers will often do this automatically for you; this file should be both human and machine readable.

Where to begin?

Package management software

Package management software usually:

  • Provides a way for you to create an environment:
    • A code project environment will contain a selection of relevant libraries, with specific versions.
    • This is kept separate from other project environments.
    • This allows you to use different versions of libraries for different projects.
  • Provides a way for you to export your environment:
    • This allows you to specify the exact libraries and library versions that were used to generate a set of results.
    • This makes it easier for others to use your work and built on it, and reproduce your code environment.
    • This saves you having to manually record library versions that you are using.

Package management software might also:

  • Help you to solve dependency conflicts:
    • Conflicts can arise when different packages depend on different versions of other packages in the background.
    • This can be tricky to resolve.
    • Remember the guideline “use dependencies wisely and sparingly” to minimise this issue.
  • Provide a way for you to package your own code into a library:
    • In addition to helping you juggle external dependencies, some package management solutions also provide a framework for turning your code into an installable package.

Depending on the language you are using, there may be many different options available. For example, for Python, you may have heard of:

  • pip
  • conda
  • pixi
  • uv
  • poetry

How do you pick?

1. License

  • Does the package manager have an open-source license?

2. Community

  • What solution does your research group use? Does your supervisor have suggestions?
  • What option can you find support for?
  • What do researchers in your field use?

3. Documentation

  • Does the package manager have easy-to-understand instructions and tutorials available?

4. Use-case

  • Are you managing all external libraries, or are you building your own packages?
  • Are you going to be pinning dependencies (like for a web-app)?1

Next steps

Where to next?

If you want some more background information and theory:

  • Return to the dependencies section of the course notes and keep reading.

If you want to learn by doing, get straight into it by following the instructions here: