# Nix > Nix package manager manual (source) from NixOS/nix on master. ## [Nix](https://docharvest.github.io/docs/nix/) Contents nix Nix Nix Nix package manager manual (source) from NixOS/nix on master. Nix package manager manual (source) from NixOS/nix on master. - [Advanced Topicsadvanced-topics](/docs/nix/advanced-topics/) - [C APIc-api](/docs/nix/c-api/) - [Command Refcommand-ref](/docs/nix/command-ref/) - [Developmentdevelopment](/docs/nix/development/) - [Glossaryglossary](/docs/nix/glossary/) - [Installationinstallation](/docs/nix/installation/) - [Introductionintroduction](/docs/nix/introduction/) - [Nix Languagelanguage](/docs/nix/language/) - [Package Managementpackage-management](/docs/nix/package-management/) - [Protocolsprotocols](/docs/nix/protocols/) - [Quick Startquick-start](/docs/nix/quick-start/) - [Nix Release Notesrelease-notes](/docs/nix/release-notes/) - [Nix Storestore](/docs/nix/store/) - [Tuning Cores and Jobsadvanced-topics/cores-vs-jobs](/docs/nix/advanced-topics/cores-vs-jobs/) - [Verifying Build Reproducibilityadvanced-topics/diff-hook](/docs/nix/advanced-topics/diff-hook/) - [Distributed Buildsadvanced-topics/distributed-builds](/docs/nix/advanced-topics/distributed-builds/) - [Using the \`eval-profiler\`advanced-topics/eval-profiler](/docs/nix/advanced-topics/eval-profiler/) - [Using the \`post-build-hook\`advanced-topics/post-build-hook](/docs/nix/advanced-topics/post-build-hook/) - [Architecturearchitecture/architecture](/docs/nix/architecture/architecture/) - [Conf File Prefixcommand-ref/conf-file-prefix](/docs/nix/command-ref/conf-file-prefix/) - [Common Environment Variablescommand-ref/env-common](/docs/nix/command-ref/env-common/) - [Experimental Commandscommand-ref/experimental-commands](/docs/nix/command-ref/experimental-commands/) - [Filescommand-ref/files](/docs/nix/command-ref/files/) - [Main Commandscommand-ref/main-commands](/docs/nix/command-ref/main-commands/) - [Nix Buildcommand-ref/nix-build](/docs/nix/command-ref/nix-build/) - [Nix Channelcommand-ref/nix-channel](/docs/nix/command-ref/nix-channel/) - [Nix Collect Garbagecommand-ref/nix-collect-garbage](/docs/nix/command-ref/nix-collect-garbage/) - [Nix Copy Closurecommand-ref/nix-copy-closure](/docs/nix/command-ref/nix-copy-closure/) - [Nix Daemoncommand-ref/nix-daemon](/docs/nix/command-ref/nix-daemon/) - [Nix Envcommand-ref/nix-env](/docs/nix/command-ref/nix-env/) - [Nix Hashcommand-ref/nix-hash](/docs/nix/command-ref/nix-hash/) - [Nix Instantiatecommand-ref/nix-instantiate](/docs/nix/command-ref/nix-instantiate/) - [Nix Prefetch Urlcommand-ref/nix-prefetch-url](/docs/nix/command-ref/nix-prefetch-url/) - [Nix Shellcommand-ref/nix-shell](/docs/nix/command-ref/nix-shell/) - [Nix Storecommand-ref/nix-store](/docs/nix/command-ref/nix-store/) - [Opt Commoncommand-ref/opt-common](/docs/nix/command-ref/opt-common/) - [Special exit codes for build failurecommand-ref/status-build-failure](/docs/nix/command-ref/status-build-failure/) - [Utilitiescommand-ref/utilities](/docs/nix/command-ref/utilities/) - [Running Benchmarksdevelopment/benchmarking](/docs/nix/development/benchmarking/) - [Building Nixdevelopment/building](/docs/nix/development/building/) …and 189 more in the sidebar. [llms.txt](/docs/nix/llms.txt) for agents ## [Advanced Topics](https://docharvest.github.io/docs/nix/advanced-topics/) Contents nix Advanced Topics Nix Advanced Topics This section lists advanced topics related to builds and builds performance ## [Tuning Cores and Jobs](https://docharvest.github.io/docs/nix/advanced-topics/cores-vs-jobs/) Contents nix Tuning Cores and Jobs Nix Tuning Cores and Jobs Nix has two relevant settings with regards to how your CPU cores will be utilized: `cores` and `max-jobs`. This chapter will talk about what they are, how they interact, and their configuration trade-offs. - `max-jobs` Dictates how many separate derivations will be built at the same time. If you set this to zero, the local machine will do no builds. Nix will still substitute from binary caches, and build remotely if remote builders are configured. - `cores` Suggests how many cores each derivation should use. Similar to `make -j`. The `cores` setting determines the value of `NIX_BUILD_CORES`. `NIX_BUILD_CORES` is equal to `cores`, unless `cores` equals `0`, in which case `NIX_BUILD_CORES` will be the total number of cores in the system. The maximum number of consumed cores is a simple multiplication, `max-jobs` \* `NIX_BUILD_CORES`. The balance on how to set these two independent variables depends upon each builder's workload and hardware. Here are a few example scenarios on a machine with 24 cores: `max-jobs` `cores` `NIX_BUILD_CORES` Maximum Processes Result 1 24 24 24 One derivation will be built at a time, each one can use 24 cores. Undersold if a job can’t use 24 cores. 4 6 6 24 Four derivations will be built at once, each given access to six cores. 12 6 6 72 12 derivations will be built at once, each given access to six cores. This configuration is over-sold. If all 12 derivations being built simultaneously try to use all six cores, the machine's performance will be degraded due to extensive context switching between the 12 builds. 24 1 1 24 24 derivations can build at the same time, each using a single core. Never oversold, but derivations which require many cores will be very slow to compile. 24 0 24 576 24 derivations can build at the same time, each using all the available cores of the machine. Very likely to be oversold, and very likely to suffer context switches. It is up to the derivations' build script to respect host's requested cores-per-build by following the value of the `NIX_BUILD_CORES` environment variable. ## [Verifying Build Reproducibility](https://docharvest.github.io/docs/nix/advanced-topics/diff-hook/) Contents nix Verifying Build Reproducibility Nix Verifying Build Reproducibility You can use Nix's `diff-hook` setting to compare build results. Note that this hook is only executed if the results differ; it is not used for determining if the results are the same. For purposes of demonstration, we'll use the following Nix file, `deterministic.nix` for testing: ``` let inherit (import {}) runCommand; in { stable = runCommand "stable" {} '' touch $out ''; unstable = runCommand "unstable" {} '' echo $RANDOM > $out ''; } ``` Additionally, `nix.conf` contains: ``` diff-hook = /etc/nix/my-diff-hook run-diff-hook = true ``` where `/etc/nix/my-diff-hook` is an executable file containing: ``` #!/bin/sh exec >&2 echo "For derivation $3:" /run/current-system/sw/bin/diff -r "$1" "$2" ``` The diff hook is executed by the same user and group who ran the build. However, the diff hook does not have write access to the store path just built. # Spot-Checking Build Determinism Verify a path which already exists in the Nix store by passing `--check` to the build command. If the build passes and is deterministic, Nix will exit with a status code of 0: ``` $ nix-build ./deterministic.nix --attr stable this derivation will be built: /nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv building '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'... /nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable $ nix-build ./deterministic.nix --attr stable --check checking outputs of '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'... /nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable ``` If the build is not deterministic, Nix will exit with a status code of 1: ``` $ nix-build ./deterministic.nix --attr unstable this derivation will be built: /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv building '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'... /nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable $ nix-build ./deterministic.nix --attr unstable --check checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'... error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs ``` In the Nix daemon's log, we will now see: ``` For derivation /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv: 1c1 < 8108 --- > 30204 ``` Using `--check` with `--keep-failed` will cause Nix to keep the second build's output in a special, `.check` path: ``` $ nix-build ./deterministic.nix --attr unstable --check --keep-failed checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'... note: keeping build directory '/tmp/nix-build-unstable.drv-0' error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs from '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check' ``` In particular, notice the `/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check` output. Nix has copied the build results to that directory where you can examine it. > \[\]{#check-dirs-are-unregistered} **Note** > > Check paths are not protected against garbage collection, and this path will be deleted on the next garbage collection. > > The path is guaranteed to be alive for the duration of the `diff-hook`'s execution, but may be deleted any time after. > > If the comparison is performed as part of automated tooling, please use the diff-hook or author your tooling to handle the case where the build was not deterministic and also a check path does not exist. `--check` is only usable if the derivation has been built on the system already. If the derivation has not been built Nix will fail with the error: ``` error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv' are not valid, so checking is not possible ``` Run the build without `--check`, and then try with `--check` again. ## [Distributed Builds](https://docharvest.github.io/docs/nix/advanced-topics/distributed-builds/) Contents nix Distributed Builds Nix Distributed Builds A local Nix installation can forward Nix builds to other machines, this allows multiple builds to be performed in parallel. Remote builds also allow Nix to perform multi-platform builds in a semi-transparent way. For example, if you perform a build for a `x86_64-darwin` on an `i686-linux` machine, Nix can automatically forward the build to a `x86_64-darwin` machine, if one is available. ## Requirements For a local machine to forward a build to a remote machine, the remote machine must: - Have Nix installed - Be running an SSH server, e.g. `sshd` - Be accessible via SSH from the local machine over the network - Have the local machine's public SSH key in `/etc/ssh/authorized_keys.d/` - Have the username of the SSH user in the `trusted-users` setting in `nix.conf` ## Testing To test connecting to a remote [Nix instance](@docroot@/glossary.md#gloss-nix-instance) (in this case `mac`), run: ``` nix store info --store ssh://username@mac ``` To specify an SSH identity file as part of the remote store URI add a query parameter, e.g. ``` nix store info --store ssh://username@mac?ssh-key=/home/alice/my-key ``` Since builds should be non-interactive, the key should not have a passphrase. Alternatively, you can load identities ahead of time into `ssh-agent` or `gpg-agent`. In a multi-user installation (default), builds are executed by the Nix Daemon. The Nix Daemon cannot prompt for a passphrase via the terminal or `ssh-agent`, so the SSH key must not have a passphrase. In addition, the Nix Daemon's user (typically root) needs to have SSH access to the remote builder. Access can be verified by running `sudo su`, and then validating SSH access, e.g. by running `ssh mac`. SSH identity files for root users are usually stored in `/root/.ssh/` (Linux) or `/var/root/.ssh` (MacOS). If you get the error ``` bash: nix: command not found error: cannot connect to 'mac' ``` then you need to ensure that the `PATH` of non-interactive login shells contains Nix. The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file. For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine: ``` uname ``` ``` Linux ``` ``` nix build --impure \ --expr '(with import { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \ --builders 'ssh://mac x86_64-darwin' ``` ``` [1/0/1 built, 0.0 MiB DL] building foo on ssh://mac ``` ``` cat ./result ``` ``` Darwin ``` It is possible to specify multiple build machines separated by a semicolon or a newline, e.g. ``` --builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd' ``` Remote build machines can also be configured in [`nix.conf`](@docroot@/command-ref/conf-file.md), e.g. ``` builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd ``` After making changes to `nix.conf`, restart the Nix daemon for changes to take effect. Finally, remote build machines can be configured in a separate configuration file included in `builders` via the syntax `@/path/to/file`. For example, ``` builders = @/etc/nix/machines ``` causes the list of machines in `/etc/nix/machines` to be included. (This is the default.) ## [Using the eval-profiler](https://docharvest.github.io/docs/nix/advanced-topics/eval-profiler/) Contents nix Using the \`eval-profiler\` Nix Using the \`eval-profiler\` Nix evaluator supports [evaluation](@docroot@/language/evaluation.md) [profiling](https://en.wikipedia.org/wiki/Profiling_\(computer_programming\)) compatible with `flamegraph.pl`. The profiler samples the nix function call stack at regular intervals. It can be enabled with the [`eval-profiler`](@docroot@/command-ref/conf-file.md#conf-eval-profiler) setting: ``` $ nix-instantiate "" -A hello --eval-profiler flamegraph ``` Stack sampling frequency and the output file path can be configured with [`eval-profile-file`](@docroot@/command-ref/conf-file.md#conf-eval-profile-file) and [`eval-profiler-frequency`](@docroot@/command-ref/conf-file.md#conf-eval-profiler-frequency). By default the collected profile is saved to `nix.profile` file in the current working directory. The collected profile can be directly consumed by `flamegraph.pl`: ``` $ flamegraph.pl nix.profile > flamegraph.svg ``` The line information in the profile contains the location of the [call site](https://en.wikipedia.org/wiki/Call_site) position and the name of the function being called (when available). For example: ``` /nix/store/2q71fdvr4h33g9832hiriwnf20fn630l-source/pkgs/top-level/default.nix:167:5:primop import ``` Here `import` primop is called at `/nix/store/2q71fdvr4h33g9832hiriwnf20fn630l-source/pkgs/top-level/default.nix:167:5`. ## [Using the post-build-hook](https://docharvest.github.io/docs/nix/advanced-topics/post-build-hook/) Contents nix Using the \`post-build-hook\` Nix Using the \`post-build-hook\` # Implementation Caveats Here we use the post-build hook to upload to a binary cache. This is a simple and working example, but it is not suitable for all use cases. The post build hook program runs after each executed build, and blocks the build loop. The build loop exits if the hook program fails. Concretely, this implementation will make Nix slow or unusable when the internet is slow or unreliable. A more advanced implementation might pass the store paths to a user-supplied daemon or queue for processing the store paths outside of the build loop. # Prerequisites This tutorial assumes you have configured an [S3-compatible binary cache](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store) as a [substituter](../command-ref/conf-file.md#conf-substituters), and that the `root` user's default AWS profile can upload to the bucket. # Set up a Signing Key Use `nix-store --generate-binary-cache-key` to create our public and private signing keys. We will sign paths with the private key, and distribute the public key for verifying the authenticity of the paths. ``` # nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public # cat /etc/nix/key.public example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM= ``` Then update [`nix.conf`](../command-ref/conf-file.md) on any machine that will access the cache. Add the cache URL to [`substituters`](../command-ref/conf-file.md#conf-substituters) and the public key to [`trusted-public-keys`](../command-ref/conf-file.md#conf-trusted-public-keys): ``` substituters = https://cache.nixos.org/ s3://example-nix-cache trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM= ``` Machines that build for the cache must sign derivations using the private key. On those machines, add the path to the key file to the [`secret-key-files`](../command-ref/conf-file.md#conf-secret-key-files) field in their [`nix.conf`](../command-ref/conf-file.md): ``` secret-key-files = /etc/nix/key.private ``` We will restart the Nix daemon in a later step. # Implementing the build hook Write the following script to `/etc/nix/upload-to-cache.sh`: ``` #!/bin/sh set -eu set -f # disable globbing export IFS=' ' echo "Uploading paths" $OUT_PATHS exec nix copy --to "s3://example-nix-cache" $OUT_PATHS ``` > **Note** > > The `$OUT_PATHS` variable is a space-separated list of Nix store paths. In this case, we expect and want the shell to perform word splitting to make each output path its own argument to `nix store sign`. Nix guarantees the paths will not contain any spaces, however a store path might contain glob characters. The `set -f` disables globbing in the shell. If you want to upload the `.drv` file too, the `$DRV_PATH` variable is also defined for the script and works just like `$OUT_PATHS`. Then make sure the hook program is executable by the `root` user: ``` # chmod +x /etc/nix/upload-to-cache.sh ``` # Updating Nix Configuration Edit `/etc/nix/nix.conf` to run our hook, by adding the following configuration snippet at the end: ``` post-build-hook = /etc/nix/upload-to-cache.sh ``` Then, restart the `nix-daemon`. # Testing Build any derivation, for example: ``` $ nix-build --expr '(import {}).writeText "example" (builtins.toString builtins.currentTime)' this derivation will be built: /nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'... running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'... post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example ``` Then delete the path from the store, and try substituting it from the binary cache: ``` $ rm ./result $ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example ``` Now, copy the path back from the cache: ``` $ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'... warning: you did not specify '--add-root'; the result might be removed by the garbage collector /nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example ``` # Conclusion We now have a Nix installation configured to automatically sign and upload every local build to a remote binary cache. Before deploying this to production, be sure to consider the [implementation caveats](#implementation-caveats). ## [Architecture](https://docharvest.github.io/docs/nix/architecture/architecture/) Contents nix Architecture Nix Architecture This chapter describes how Nix works. It should help users understand why Nix behaves as it does, and it should help developers understand how to modify Nix and how to write similar tools. ## Overview Nix consists of [hierarchical layers](https://en.wikipedia.org/wiki/Multitier_architecture#Layers). The following [concept map](https://en.wikipedia.org/wiki/Concept_map) shows its main components (rectangles), the objects they operate on (rounded rectangles), and their interactions (connecting phrases): ``` .----------------. | Nix expression |----------. '----------------' | | passed to | | +----------|-------------------|--------------------------------+ | Nix | V | | | +------------------------+ | | | | command line interface |------. | | | +------------------------+ | | | | | | | | evaluated by calls manages | | | | | | | | V | | | | +--------------------+ | | | '-------->| language evaluator | | | | +--------------------+ | | | | | | | produces | | | | V | | +----------------------------|------------------------------+ | | | store | | | | | referenced by V builds | | | | .-------------. .------------. .--------------. | | | | | build input |----->| build plan |----->| build result | | | | | '-------------' '------------' '--------------' | | | +-------------------------------------------------|---------+ | +---------------------------------------------------|-----------+ | represented as | V .---------------. | file | '---------------' ``` At the top is the [command line interface](../command-ref/index.md) that drives the underlying layers. The [Nix language](../language/index.md) evaluator transforms Nix expressions into self-contained _build plans_, which are used to derive _build results_ from referenced _build inputs_. The command line interface and Nix expressions are what users deal with most. > **Note** > > The Nix language itself does not have a notion of _packages_ or _configurations_. As far as we are concerned here, the inputs and results of a build plan are just data. Underlying the command line interface and the Nix language evaluator is the [Nix store](../store/index.md), a mechanism to keep track of build plans, data, and references between them. It can also execute build plans to produce new data, which are made available to the operating system as files. A build plan itself is a series of _build tasks_, together with their build inputs. > **Important** A build task in Nix is called [store derivation](@docroot@/glossary.md#gloss-store-derivation). Each build task has a special build input executed as _build instructions_ in order to perform the build. The result of a build task can be input to another build task. The following [data flow diagram](https://en.wikipedia.org/wiki/Data-flow_diagram) shows a build plan for illustration. Build inputs used as instructions to a build task are marked accordingly: ``` +--------------------------------------------------------------------+ | build plan | | | | .-------------. | | | build input |---------. | | '-------------' | | | instructions | | | | | v | | .-------------. .----------. | | | build input |-->( build task )-------. | | '-------------' '----------' | | | instructions | | | | | v | | .-------------. .----------. .--------------. | | | build input |---------. ( build task )--->| build result | | | '-------------' | '----------' '--------------' | | instructions ^ | | | | | | v | | | .-------------. .----------. | | | | build input |-->( build task )-------' | | '-------------' '----------' | | ^ | | | | | | | | .-------------. | | | | build input |---------' | | '-------------' | | | +--------------------------------------------------------------------+ ``` ## [C API](https://docharvest.github.io/docs/nix/c-api/) Contents nix C API Nix C API Nix provides a C API with the intent of [_becoming_](https://github.com/NixOS/nix/milestone/52) a stable API, which it is currently not. It is in development. See: - C API documentation for a recent build of master - [Getting Started](https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs) - [Index](https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs/globals.html) - [Matrix Room _Nix Bindings_](https://matrix.to/#/#nix-bindings:nixos.org) for discussion and questions. - [Stabilisation Milestone](https://github.com/NixOS/nix/milestone/52) - [Other C API PRs and issues](https://github.com/NixOS/nix/labels/c%20api) - [Contributing C API Documentation](development/documentation.md#c-api-documentation), including how to build it locally. ## [Command Ref](https://docharvest.github.io/docs/nix/command-ref/) Contents nix Command Ref Nix Command Ref This section lists commands and options that you can use when you work with Nix. ## [Conf File Prefix](https://docharvest.github.io/docs/nix/command-ref/conf-file-prefix/) Contents nix Conf File Prefix Nix Conf File Prefix `nix.conf` - Nix configuration file # Description Nix supports a variety of configuration settings, which are read from configuration files or taken as command line flags. ## Configuration file By default Nix reads settings from each of the following places. Settings are applied on top of one another, so later settings overwrite earlier entries. 1. The system-wide configuration file `nix.conf` in the configuration directory. The configuration directory defaults to `${sysconfdir}/nix` (i.e. `/etc/nix` on most Unix systems) or `%PROGRAMDATA%\nix\conf` on Windows, and can be overridden with [`NIX_CONF_DIR`](./env-common.md#env-NIX_CONF_DIR). Values loaded in this file are not forwarded to the Nix daemon. The client assumes that the daemon has already loaded them. 2. If [`NIX_USER_CONF_FILES`](./env-common.md#env-NIX_USER_CONF_FILES) is set, then each path in the list will be loaded in reverse order. The list separator is `:` on Unix and `;` on Windows. Otherwise it will look for `nix.conf` in: - The [user configuration directory](./env-common.md#user-conf-dir) - On Unix additionally, `nix/nix.conf` under each directory in [`XDG_CONFIG_DIRS`](./env-common.md#env-XDG_CONFIG_DIRS) 3. If [`NIX_CONFIG`](./env-common.md#env-NIX_CONFIG) is set, its contents are treated as the contents of a configuration file. In particular, settings are separated by newlines, just as in the configuration file. ### File format Configuration files consist of `name = value` pairs, one per line. Comments start with a `#` character. Example: ``` keep-outputs = true # Nice for developers keep-derivations = true # Idem ``` Other files can be included with a line like `include `, where `` is interpreted relative to the current configuration file. A missing file is an error unless `!include` is used instead. A configuration setting usually overrides any previous value. However, for settings that take a list of items, you can prefix the name of the setting by `extra-` to _append_ to the previous value. For instance, ``` substituters = a b extra-substituters = c d ``` defines the `substituters` setting to be `a b c d`. Unknown option names are not an error, and are simply ignored with a warning. ## Command line flags Configuration options can be set on the command line, overriding the values set in the [configuration file](#configuration-file): - Every configuration setting has corresponding command line flag (e.g. `--max-jobs 16`). Boolean settings do not need an argument, and can be explicitly disabled with the `no-` prefix (e.g. `--keep-failed` and `--no-keep-failed`). Unknown option names are invalid flags (unless there is already a flag with that name), and are rejected with an error. - The flag `--option ` is interpreted exactly like a ` = ` in a setting file. Unknown option names are ignored with a warning. The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`). ## Integer settings Settings that have an integer type support the suffixes `K`, `M`, `G` and `T`. These cause the specified value to be multiplied by 2^10, 2^20, 2^30 and 2^40, respectively. For instance, `--min-free 1M` is equivalent to `--min-free 1048576`. # Available settings ## [Common Environment Variables](https://docharvest.github.io/docs/nix/command-ref/env-common/) Contents nix Common Environment Variables Nix Common Environment Variables Most Nix commands interpret the following environment variables. ## Configuration environment variables The following environment variables affect how Nix loads its configuration. See the [configuration file](@docroot@/command-ref/conf-file.md#configuration-file) documentation for full details. - [`NIX_CONF_DIR`](#env-NIX_CONF_DIR) Overrides the system configuration directory. - [`NIX_USER_CONF_FILES`](#env-NIX_USER_CONF_FILES) Overrides the user configuration file locations. - [`NIX_CONFIG`](#env-NIX_CONFIG) Provides configuration settings inline. ## Store setting environment variables - [`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE) Normally, the Nix store directory (typically `/nix/store`) is not allowed to contain any symlink components. This is to prevent “impure” builds. Builders sometimes “canonicalise” paths by resolving all symlink components. Thus, builds on different machines (with `/nix/store` resolving to different locations) could yield different results. This is generally not a problem, except when builds are deployed to machines where `/nix/store` resolves differently. If you are sure that you’re not going to do that, you can set `NIX_IGNORE_SYMLINK_STORE` to `1`. Note that if you’re symlinking the Nix store so that you can put it on another file system than the root file system, on Linux you’re better off using `bind` mount points, e.g., ``` $ mkdir /nix $ mount -o bind /mnt/otherdisk/nix /nix ``` Consult the mount 8 manual page for details. - [`NIX_STORE_DIR`](#env-NIX_STORE_DIR) Overrides the location of the Nix store. On Unix, the default is `/nix/store`, set at compile time by the `libstore:store-dir` build option. On Windows, there is no compile-time setting; the default is `%PROGRAMDATA%\nix\store`, determined at runtime using the [`%PROGRAMDATA%` Known Folder](#known-folders). See the [Store Types](@docroot@/store/types/index.md) chapter; each store has a `store` setting linking back to this as its default. - [`NIX_LOG_DIR`](#env-NIX_LOG_DIR) Overrides the location of the Nix log directory. On Unix, the default is `/nix/var/log/nix`, set at compile time by the `log-dir` build option. On Windows, there is no compile-time setting; the default is `%PROGRAMDATA%\nix\log`, determined at runtime using the [`%PROGRAMDATA%` Known Folder](#known-folders). The [Local Store](@docroot@/store/types/local-store.md), [Local Daemon Store](@docroot@/store/types/local-daemon-store.md), and [Experimental SSH Store with filesystem mounted](@docroot@/store/types/experimental-ssh-store-with-filesystem-mounted.md) have per-store settings that override this. - [`NIX_STATE_DIR`](#env-NIX_STATE_DIR) Overrides the location of the Nix state directory. On Unix, the default is `${localstatedir}/nix`, where `localstatedir` is a compile-time build option defaulting to `/nix/var`. On Windows, there is no compile-time setting; the default is `%PROGRAMDATA%\nix\state`, determined at runtime using the [`%PROGRAMDATA%` Known Folder](#known-folders). The [Local Store](@docroot@/store/types/local-store.md), [Local Daemon Store](@docroot@/store/types/local-daemon-store.md), and [Experimental SSH Store with filesystem mounted](@docroot@/store/types/experimental-ssh-store-with-filesystem-mounted.md) have per-store settings that override this. - [`NIX_DAEMON_SOCKET_PATH`](#env-NIX_DAEMON_SOCKET_PATH) Overrides the path to the Unix domain socket used to communicate with the Nix daemon. Defaults to `daemon-socket/socket` within the state directory (see [`NIX_STATE_DIR`](#env-NIX_STATE_DIR)). See the [Local Daemon Store](@docroot@/store/types/local-daemon-store.md) documentation for details on how the socket path is resolved. - [`TMPDIR`](#env-TMPDIR) Use the specified directory to store temporary files. In particular, this includes temporary build directories; these can take up substantial amounts of disk space. The default is `/tmp`. - [`NIX_REMOTE`](#env-NIX_REMOTE) Overrides the [`store`](@docroot@/command-ref/conf-file.md#conf-store) setting. ## Nix language evaluator settings - [`NIX_SHOW_STATS`](#env-NIX_SHOW_STATS) If set to `1`, Nix will print some evaluation statistics, such as the number of values allocated. - [`NIX_COUNT_CALLS`](#env-NIX_COUNT_CALLS) If set to `1`, Nix will print how often functions were called during Nix expression evaluation. This is useful for profiling your Nix expressions. - [`GC_INITIAL_HEAP_SIZE`](#env-GC_INITIAL_HEAP_SIZE) If Nix has been configured to use the Boehm garbage collector, this variable sets the initial size of the heap in bytes. It defaults to 384 MiB. Setting it to a low value reduces memory consumption, but will increase runtime due to the overhead of garbage collection. - [`NIX_PATH`](#env-NIX_PATH) A colon-separated list of search path entries used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md). This environment variable overrides the value of the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path). It can be extended using the [`-I` option](@docroot@/command-ref/opt-common.md#opt-I). > **Example** > > ``` > $ export NIX_PATH=/home/eelco/Dev:nixos-config=/etc/nixos > ``` If `NIX_PATH` is set to an empty string, resolving search paths will always fail. > **Example** > > ``` > $ NIX_PATH= nix-instantiate --eval '' > error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I) > ``` ## User Directories Nix supports per-user **configuration**, **state**, and **cache** directories. The following environment variables override the locations for these directories: - \[user configuration directory\]{#user-conf-dir}: \[`NIX_CONFIG_HOME`\]{#env-NIX\_CONFIG\_HOME} - user state directory: \[`NIX_STATE_HOME`\]{#env-NIX\_STATE\_HOME} - user cache directory: \[`NIX_CACHE_HOME`\]{#env-NIX\_CACHE\_HOME} When these are not set, the defaults depend on the platform: - On Unix, the [XDG base directories](#xdg-base-directories): `$XDG_CONFIG_HOME/nix`, `$XDG_STATE_HOME/nix`, `$XDG_CACHE_HOME/nix` - On Windows, the [Windows Known Folders](#known-folders): `%APPDATA%\nix\config`, `%LOCALAPPDATA%\nix\state`, `%LOCALAPPDATA%\nix\cache` For backwards compatibility, legacy Nix commands (e.g. `nix-env`, `nix-channel`) use dotfiles in `$HOME` instead of these directories unless [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is enabled. [New Nix commands](@docroot@/command-ref/new-cli/nix.md) (experimental) use the proper directories by default. When [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is enabled, the configuration directory is resolved as: 1. `$NIX_CONFIG_HOME`, if it is defined 2. Otherwise, the platform default (e.g. `$XDG_CONFIG_HOME/nix` on Unix) Likewise for the state and cache directories. ## Miscellaneous environment variables - [`IN_NIX_SHELL`](#env-IN_NIX_SHELL) Indicator that tells if the current environment was set up by `nix-shell`. It can have the values `pure` or `impure`. ## Appendix: OS-specific conventions This information is not Nix-specific, but is referenced above. ### Unix: XDG Base Directories {#xdg-base-directories} The [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) defines standard locations for user-specific configuration, state, and cache files on Unix systems. The following environment variables are used: - \[`XDG_CONFIG_HOME`\]{#env-XDG\_CONFIG\_HOME} (default `~/.config`) - \[`XDG_STATE_HOME`\]{#env-XDG\_STATE\_HOME} (default `~/.local/state`) - \[`XDG_CACHE_HOME`\]{#env-XDG\_CACHE\_HOME} (default `~/.cache`) - \[`XDG_CONFIG_DIRS`\]{#env-XDG\_CONFIG\_DIRS} (default `/etc/xdg`) — colon-separated list of additional configuration base directories, searched after `XDG_CONFIG_HOME` ### Windows: Known Folders {#known-folders} On Windows, [Known Folders](https://learn.microsoft.com/en-us/windows/win32/shell/known-folders) provide standard locations for application data on Windows. The relevant folders are: - \[`%APPDATA%`\]{#env-APPDATA} — per-user roaming application data - \[`%LOCALAPPDATA%`\]{#env-LOCALAPPDATA} — per-user local application data - \[`%PROGRAMDATA%`\]{#env-PROGRAMDATA} — system-wide application data ## [Experimental Commands](https://docharvest.github.io/docs/nix/command-ref/experimental-commands/) Contents nix Experimental Commands Nix Experimental Commands This section lists [experimental commands](@docroot@/development/experimental-features.md#xp-feature-nix-command). > **Warning** > > These commands may be removed in the future, or their syntax may change in incompatible ways. ## [Files](https://docharvest.github.io/docs/nix/command-ref/files/) Contents nix Files Nix Files This section lists configuration files that you can use when you work with Nix. ## [Channels](https://docharvest.github.io/docs/nix/command-ref/files/channels/) Contents nix Channels Nix Channels ## Channels A directory containing symlinks to Nix channels, managed by [`nix-channel`](@docroot@/command-ref/nix-channel.md): - `$XDG_STATE_HOME/nix/profiles/channels` for regular users - `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root` [`nix-channel`](@docroot@/command-ref/nix-channel.md) uses a [profile](@docroot@/command-ref/files/profiles.md) to store channels. This profile contains symlinks to the contents of those channels. ## Subscribed channels The list of subscribed channels is stored in - `~/.nix-channels` - `$XDG_STATE_HOME/nix/channels` if [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is set to `true` in the following format: ``` ... ``` ## [Default Nix Expression](https://docharvest.github.io/docs/nix/command-ref/files/default-nix-expression/) Contents nix Default Nix Expression Nix Default Nix Expression ## Default Nix expression The source for the [Nix expressions](@docroot@/glossary.md#gloss-nix-expression) used by [`nix-env`](@docroot@/command-ref/nix-env.md) by default: - `~/.nix-defexpr` - `$XDG_STATE_HOME/nix/defexpr` if [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is set to `true`. It is loaded as follows: - If the default expression is a file, it is loaded as a Nix expression. - If the default expression is a directory containing a `default.nix` file, that `default.nix` file is loaded as a Nix expression. - If the default expression is a directory without a `default.nix` file, then its contents (both files and subdirectories) are loaded as Nix expressions. The expressions are combined into a single attribute set, each expression under an attribute with the same name as the original file or subdirectory. Subdirectories without a `default.nix` file are traversed recursively in search of more Nix expressions, but the names of these intermediate directories are not added to the attribute paths of the default Nix expression. Then, the resulting expression is interpreted like this: - If the expression is an attribute set, it is used as the default Nix expression. - If the expression is a function, an empty set is passed as argument and the return value is used as the default Nix expression. > **Example** > > If the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to > > ``` > { > foo = import ~/.nix-defexpr/foo.nix; > bar = import ~/.nix-defexpr/bar.nix; > } > ``` The file [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) is always ignored. The command [`nix-channel`](@docroot@/command-ref/nix-channel.md) places a symlink to the current user's [channels](@docroot@/command-ref/files/channels.md) in this directory, the [user channel link](#user-channel-link). This makes all subscribed channels available as attributes in the default expression. ## User channel link A symlink that ensures that [`nix-env`](@docroot@/command-ref/nix-env.md) can find the current user's [channels](@docroot@/command-ref/files/channels.md): - `~/.nix-defexpr/channels` - `$XDG_STATE_HOME/nix/defexpr/channels` if [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is set to `true`. This symlink points to: - `$XDG_STATE_HOME/nix/profiles/channels` for regular users - `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root` In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user. ## [Profiles](https://docharvest.github.io/docs/nix/command-ref/files/profiles/) Contents nix Profiles Nix Profiles ## Profiles A directory that contains links to profiles managed by [`nix-env`](@docroot@/command-ref/nix-env.md) and [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md): - `$XDG_STATE_HOME/nix/profiles` for regular users - `$NIX_STATE_DIR/profiles/per-user/root` if the user is `root` A profile is a directory of symlinks to files in the Nix store. ### Filesystem layout Profiles are versioned as follows. When using a profile named _path_, _path_ is a symlink to _path_`-`_N_`-link`, where _N_ is the version of the profile. In turn, _path_`-`_N_`-link` is a symlink to a path in the Nix store. For example: ``` $ ls -l ~alice/.local/state/nix/profiles/profile* lrwxrwxrwx 1 alice users 14 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile -> profile-7-link lrwxrwxrwx 1 alice users 51 Oct 28 16:18 /home/alice/.local/state/nix/profiles/profile-5-link -> /nix/store/q69xad13ghpf7ir87h0b2gd28lafjj1j-profile lrwxrwxrwx 1 alice users 51 Oct 29 13:20 /home/alice/.local/state/nix/profiles/profile-6-link -> /nix/store/6bvhpysd7vwz7k3b0pndn7ifi5xr32dg-profile lrwxrwxrwx 1 alice users 51 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile-7-link -> /nix/store/mp0x6xnsg0b8qhswy6riqvimai4gm677-profile ``` Each of these symlinks is a root for the Nix garbage collector. The contents of the store path corresponding to each version of the profile is a tree of symlinks to the files of the installed packages, e.g. ``` $ ll -R ~eelco/.local/state/nix/profiles/profile-7-link/ /home/eelco/.local/state/nix/profiles/profile-7-link/: total 20 dr-xr-xr-x 2 root root 4096 Jan 1 1970 bin -r--r--r-- 2 root root 1402 Jan 1 1970 manifest.nix dr-xr-xr-x 4 root root 4096 Jan 1 1970 share /home/eelco/.local/state/nix/profiles/profile-7-link/bin: total 20 lrwxrwxrwx 5 root root 79 Jan 1 1970 chromium -> /nix/store/cyxny9d1zjb9l9103fr6j6kavp3bqjxf-chromium-86.0.4240.111/bin/chromium lrwxrwxrwx 7 root root 87 Jan 1 1970 spotify -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/bin/spotify lrwxrwxrwx 3 root root 79 Jan 1 1970 zoom-us -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/bin/zoom-us /home/eelco/.local/state/nix/profiles/profile-7-link/share/applications: total 12 lrwxrwxrwx 4 root root 120 Jan 1 1970 chromium-browser.desktop -> /nix/store/sqzyx2l85i6j2a77pnyvglh3bvzwmjjp-chromium-unwrapped-86.0.4240.111/share/applications/chromium-browser.desktop lrwxrwxrwx 7 root root 110 Jan 1 1970 spotify.desktop -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/share/applications/spotify.desktop lrwxrwxrwx 3 root root 107 Jan 1 1970 us.zoom.Zoom.desktop -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/share/applications/us.zoom.Zoom.desktop … ``` Each profile version contains a manifest file: - [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) used by [`nix-env`](@docroot@/command-ref/nix-env.md). - [`manifest.json`](@docroot@/command-ref/files/manifest.json.md) used by [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md) (experimental). ## User profile link A symbolic link to the user's current profile: - `~/.nix-profile` - `$XDG_STATE_HOME/nix/profile` if [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories) is set to `true`. By default, this symlink points to: - `$XDG_STATE_HOME/nix/profiles/profile` for regular users - `$NIX_STATE_DIR/profiles/per-user/root/profile` for `root` The `PATH` environment variable should include `/bin` subdirectory of the profile link (e.g. `~/.nix-profile/bin`) for the user environment to be visible to the user. The [installer](@docroot@/installation/installing-binary.md) sets this up by default, unless you enable [`use-xdg-base-directories`](@docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories). ## [Main Commands](https://docharvest.github.io/docs/nix/command-ref/main-commands/) Contents nix Main Commands Nix Main Commands This section lists commands and options that you can use when you work with Nix. ## [Nix Build](https://docharvest.github.io/docs/nix/command-ref/nix-build/) ## [Nix Channel](https://docharvest.github.io/docs/nix/command-ref/nix-channel/) Contents nix Nix Channel Nix Nix Channel `nix-channel` - manage Nix channels # Synopsis `nix-channel` {`--add` url \[_name_\] | `--remove` _name_ | `--list` | `--update` \[_names…_\] | `--list-generations` | `--rollback` \[_generation_\] } # Description Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version. The moving parts of channels are: - The official channels listed at [https://channels.nixos.org](https://channels.nixos.org) - The user-specific list of [subscribed channels](#subscribed-channels) - The [downloaded channel contents](#channels) - The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH) > **Note** > > The state of a subscribed channel is external to the Nix expressions relying on it. This may limit reproducibility. > > Dependencies on other Nix expressions can be declared explicitly with: > > - [`fetchurl`](@docroot@/language/builtins.md#builtins-fetchurl), [`fetchTarball`](@docroot@/language/builtins.md#builtins-fetchTarball), or [`fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) in Nix expressions > - the [`-I` option](@docroot@/command-ref/opt-common.md#opt-I) in command line invocations This command has the following operations: - `--add` _url_ \[_name_\] Add a channel _name_ located at _url_ to the list of subscribed channels. If _name_ is omitted, default to the last component of _url_, with the suffixes `-stable` or `-unstable` removed. > **Note** > > `--add` does not automatically perform an update. Use `--update` explicitly. A channel URL must point to a directory containing a file `nixexprs.tar.gz`. At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point. - `--remove` _name_ Remove the channel _name_ from the list of subscribed channels. - `--list` Print the names and URLs of all subscribed channels on standard output. - `--update` \[_names_…\] Download the Nix expressions of subscribed channels and create a new generation. Update all channels if none is specified, and only those included in _names_ otherwise. > **Note** > > Downloaded channel contents are cached. Use `--tarball-ttl` or the [`tarball-ttl` configuration option](@docroot@/command-ref/conf-file.md#conf-tarball-ttl) to change the validity period of cached downloads. - `--list-generations` Prints a list of all the current existing generations for the channel profile. Works the same way as ``` nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations ``` - `--rollback` \[_generation_\] Revert channels to the state before the last call to `nix-channel --update`. Optionally, you can specify a specific channel _generation_ number to restore. {{#include ./opt-common.md}} {{#include ./env-common.md}} # Files `nix-channel` operates on the following files. {{#include ./files/channels.md}} # Examples Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package: ``` $ nix-channel --add https://channels.nixos.org/nixpkgs-unstable $ nix-channel --list nixpkgs https://channels.nixos.org/nixpkgs $ nix-channel --update $ nix-shell -p hello --run hello hello ``` Revert channel updates using `--rollback`: ``` $ nix-instantiate --eval '' --attr lib.version "22.11pre296212.530a53dcbc9" $ nix-channel --rollback switching from generation 483 to 482 $ nix-instantiate --eval '' --attr lib.version "22.11pre281526.d0419badfad" ``` Remove a channel: ``` $ nix-channel --remove nixpkgs $ nix-channel --list ``` ## [Nix Collect Garbage](https://docharvest.github.io/docs/nix/command-ref/nix-collect-garbage/) Contents nix Nix Collect Garbage Nix Nix Collect Garbage `nix-collect-garbage` - delete unreachable [store objects](@docroot@/store/store-object.md) # Synopsis `nix-collect-garbage` \[`--delete-old`\] \[`-d`\] \[`--delete-older-than` _period_\] \[`--max-freed` _bytes_\] \[`--dry-run`\] # Description The command `nix-collect-garbage` is mostly an alias of [`nix-store --gc`](@docroot@/command-ref/nix-store/gc.md). That is, it deletes all unreachable [store objects](@docroot@/store/store-object.md) in the Nix store to clean up your system. However, it provides two additional options, [`--delete-old`](#opt-delete-old) and [`--delete-older-than`](#opt-delete-older-than), which also delete old [profiles](@docroot@/command-ref/files/profiles.md), allowing potentially more [store objects](@docroot@/store/store-object.md) to be deleted because profiles are also garbage collection roots. These options are the equivalent of running [`nix-env --delete-generations`](@docroot@/command-ref/nix-env/delete-generations.md) with various augments on multiple profiles, prior to running `nix-collect-garbage` (or just `nix-store --gc`) without any flags. > **Note** > > Deleting previous configurations makes rollbacks to them impossible. These flags should be used with care, because they potentially delete generations of profiles used by other users on the system. ## Locations searched for profiles `nix-collect-garbage` cannot know about all profiles; that information doesn't exist. Instead, it looks in a few locations, and acts on all profiles it finds there: 1. The default profile locations as specified in the [profiles](@docroot@/command-ref/files/profiles.md) section of the manual. 2. > **NOTE** > > Not stable; subject to change > > Do not rely on this functionality; it just exists for migration purposes and may change in the future. These deprecated paths remain a private implementation detail of Nix. `$NIX_STATE_DIR/profiles` and `$NIX_STATE_DIR/profiles/per-user`. With the exception of `$NIX_STATE_DIR/profiles/per-user/root` and `$NIX_STATE_DIR/profiles/default`, these directories are no longer used by other commands. `nix-collect-garbage` looks there anyways in order to clean up profiles from older versions of Nix. # Options These options are for deleting old [profiles](@docroot@/command-ref/files/profiles.md) prior to deleting unreachable [store objects](@docroot@/store/store-object.md). - [`--delete-old`](#opt-delete-old) / `-d` Delete all old generations of profiles. This is the equivalent of invoking [`nix-env --delete-generations old`](@docroot@/command-ref/nix-env/delete-generations.md#generations-old) on each found profile. - [`--delete-older-than`](#opt-delete-older-than) _period_ Delete all generations of profiles older than the specified amount (except for the generations that were active at that point in time). _period_ is a value such as `30d`, which would mean 30 days. This is the equivalent of invoking [`nix-env --delete-generations `](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile. See the documentation of that command for additional information about the _period_ argument. - [`--max-freed`](#opt-max-freed) _bytes_ Keep deleting paths until at least _bytes_ bytes have been deleted, then stop. The argument _bytes_ can be followed by the multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB or TiB units. {{#include ./opt-common.md}} {{#include ./env-common.md}} # Example To delete from the Nix store everything that is not used by the current generations of each profile, do ``` $ nix-collect-garbage -d ``` ## [Nix Copy Closure](https://docharvest.github.io/docs/nix/command-ref/nix-copy-closure/) Contents nix Nix Copy Closure Nix Nix Copy Closure `nix-copy-closure` - copy store objects to or from a remote machine via SSH # Synopsis `nix-copy-closure` \[`--to` | `--from` \] \[`--gzip`\] \[`--include-outputs`\] \[`--use-substitutes` | `-s`\] \[`-v`\] \[_user_@\]_machine_\[:_port_\] _paths_ # Description Given _paths_ from one machine, `nix-copy-closure` computes the [closure](@docroot@/glossary.md#gloss-closure) of those paths (i.e. all their dependencies in the Nix store), and copies [store objects](@docroot@/glossary.md#gloss-store-object) in that closure to another machine via SSH. It doesn’t copy store objects that are already present on the other machine. > **Note** > > While the Nix store to use on the local machine can be specified on the command line with the [`--store`](@docroot@/command-ref/conf-file.md#conf-store) option, the Nix store to be accessed on the remote machine can only be [configured statically](@docroot@/command-ref/conf-file.md#configuration-file) on that remote machine. Since `nix-copy-closure` calls `ssh`, you may need to authenticate with the remote machine. In fact, you may be asked for authentication _twice_ because `nix-copy-closure` currently connects twice to the remote machine: first to get the set of paths missing on the target machine, and second to send the dump of those paths. When using public key authentication, you can avoid typing the passphrase with `ssh-agent`. # Options - `--to` Copy the closure of _paths_ from a Nix store accessible from the local machine to the Nix store on the remote _machine_. This is the default behavior. - `--from` Copy the closure of _paths_ from the Nix store on the remote _machine_ to the local machine's specified Nix store. - `--gzip` Enable compression of the SSH connection. - `--include-outputs` Also copy the outputs of [store derivation](@docroot@/glossary.md#gloss-store-derivation)s included in the closure. - `--use-substitutes` / `-s` Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters). Any store objects that cannot be substituted on the target are still copied normally from the source. This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast. {{#include ./opt-common.md}} # Environment variables - `NIX_SSHOPTS` Additional options to be passed to `ssh` on the command line. {{#include ./env-common.md}} # Examples > **Example** > > Copy GNU Hello with all its dependencies to a remote machine: > > ``` > $ storePath="$(nix-build '' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link)" > $ nix-copy-closure --to alice@itchy.example.org "$storePath" > copying 5 paths... > copying path '/nix/store/h6q8sqsqfbd3252f9gixqn3z282wds7m-xgcc-13.2.0-libgcc' to 'ssh://alice@itchy.example.org'... > copying path '/nix/store/imnwvn96lw355giswsk36hx105j4wnpj-libunistring-1.1' to 'ssh://alice@itchy.example.org'... > copying path '/nix/store/85301indj7scg34spnfczkz72jgv8wa9-libidn2-2.3.7' to 'ssh://alice@itchy.example.org'... > copying path '/nix/store/ypwfsaljwhzw9iffiysxmxnhjj8v7np0-glibc-2.39-31' to 'ssh://alice@itchy.example.org'... > copying path '/nix/store/0dklv59zppdsqdvgf0qdvjgzcs5wbwxa-hello-2.12.1' to 'ssh://alice@itchy.example.org'... > ``` > **Example** > > Copy GNU Hello from a remote machine using a known store path, and run it: > > ``` > $ storePath="$(nix-instantiate --eval --raw '' -I nixpkgs=channel:nixpkgs-unstable -A hello.outPath)" > $ nix-copy-closure --from alice@itchy.example.org "$storePath" > $ "$storePath"/bin/hello > Hello, world! > ``` ## [Nix Daemon](https://docharvest.github.io/docs/nix/command-ref/nix-daemon/) Contents nix Nix Daemon Nix Nix Daemon `nix-daemon` - Nix multi-user support daemon # Synopsis `nix-daemon` # Description The Nix daemon is necessary in multi-user Nix installations. It runs build tasks and other operations on the Nix store on behalf of unprivileged users. ## [Nix Env](https://docharvest.github.io/docs/nix/command-ref/nix-env/) Contents nix Nix Env Nix Nix Env `nix-env` - manipulate or query Nix user environments # Synopsis `nix-env` _operation_ \[_options_\] \[_arguments…_\] \[`--option` _name_ _value_\] \[`--arg` _name_ _value_\] \[`--argstr` _name_ _value_\] \[{`--file` | `-f`} _path_\] \[{`--profile` | `-p`} _path_\] \[`--system-filter` _system_\] \[`--dry-run`\] # Description The command `nix-env` is used to manipulate Nix user environments. User environments are sets of software packages available to a user at some point in time. In other words, they are a synthesised view of the programs available in the Nix store. There may be many user environments: different users can have different environments, and individual users can switch between different environments. `nix-env` takes exactly one _operation_ flag which indicates the subcommand to be performed. The following operations are available: - [`--install`](./nix-env/install.md) - [`--upgrade`](./nix-env/upgrade.md) - [`--uninstall`](./nix-env/uninstall.md) - [`--set`](./nix-env/set.md) - [`--set-flag`](./nix-env/set-flag.md) - [`--query`](./nix-env/query.md) - [`--switch-profile`](./nix-env/switch-profile.md) - [`--list-generations`](./nix-env/list-generations.md) - [`--delete-generations`](./nix-env/delete-generations.md) - [`--switch-generation`](./nix-env/switch-generation.md) - [`--rollback`](./nix-env/rollback.md) These pages can be viewed offline: - `man nix-env-`. Example: `man nix-env-install` - `nix-env --help --` Example: `nix-env --help --install` # Package sources `nix-env` can obtain packages from multiple sources: - An attribute set of derivations from: - The [default Nix expression](@docroot@/command-ref/files/default-nix-expression.md) (by default) - A Nix file, specified via `--file` - A [profile](@docroot@/command-ref/files/profiles.md), specified via `--from-profile` - A Nix expression that is a function which takes default expression as argument, specified via `--from-expression` - A [store path](@docroot@/store/store-path.md) # Selectors Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of _arguments_ that specify the packages on which to operate. Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name): - `name`: Everything up to but not including the first dash (`-`) that is _not_ followed by a letter. - `version`: The rest, excluding the separating dash. > **Example** > > `nix-env` parses the symbolic derivation name `apache-httpd-2.0.48` as: > > ``` > { > "name": "apache-httpd", > "version": "2.0.48" > } > ``` > **Example** > > `nix-env` parses the symbolic derivation name `firefox.*` as: > > ``` > { > "name": "firefox.*", > "version": "" > } > ``` The `name` parts of the _arguments_ to `nix-env` are treated as extended regular expressions and matched against the `name` parts of derivation names in the package source. The match is case-sensitive. The regular expression can optionally be followed by a dash (`-`) and a version number; if omitted, any version of the package will match. For details on regular expressions, see [**regex**(7)](https://linux.die.net/man/7/regex). > **Example** > > Common patterns for finding package names with `nix-env`: > > - `firefox` > > Matches the package name `firefox` and any version. > > - `firefox-32.0` > > Matches the package name `firefox` and version `32.0`. > > - `gtk\\+` > > Matches the package name `gtk+`. The `+` character must be escaped using a backslash (`\`) to prevent it from being interpreted as a quantifier, and the backslash must be escaped in turn with another backslash to ensure that the shell passes it on. > > - `.\*` > > Matches any package name. This is the default for most commands. > > - `'.*zip.*'` > > Matches any package name containing the string `zip`. Note the dots: `'*zip*'` does not work, because in a regular expression, the character `*` is interpreted as a quantifier. > > - `'.*(firefox|chromium).*'` > > Matches any package name containing the strings `firefox` or `chromium`. > # Files `nix-env` operates on the following files. {{#include ./files/default-nix-expression.md}} {{#include ./files/profiles.md}} ## [Delete Generations](https://docharvest.github.io/docs/nix/command-ref/nix-env/delete-generations/) Contents nix Delete Generations Nix Delete Generations `nix-env --delete-generations` - delete profile generations # Synopsis `nix-env` `--delete-generations` _generations_ # Description This operation deletes the specified generations of the current profile. _generations_ can be a one of the following: - [`...`](#generations-list) A list of generation numbers, each one a separate command-line argument. Delete exactly the profile generations given by their generation number. Deleting the current generation is not allowed. - [The special value `old`](#generations-old) Delete all generations except the current one. > **WARNING** > > Older _and newer_ generations will be deleted by this operation. > > One might expect this to just delete older generations than the current one, but that is only true if the current generation is also the latest. Because one can roll back to a previous generation, it is possible to have generations newer than the current one. They will also be deleted. - [`d`](#generations-time) The last _number_ days _Example_: `30d` Delete all generations created more than _number_ days ago, except the most recent one of them. This allows rolling back to generations that were available within the specified period. - [`+`](#generations-count) The last _number_ generations up to the present _Example_: `+5` Keep the last _number_ generations, along with any newer than current. Periodically deleting old generations is important to make garbage collection effective. The is because profiles are also garbage collection roots — any [store object](@docroot@/store/store-object.md) reachable from a profile is "alive" and ineligible for deletion. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ## Delete explicit generation numbers ``` $ nix-env --delete-generations 3 4 8 ``` Delete the generations numbered 3, 4, and 8, so long as the current active generation is not any of those. ## Keep most-recent by count (number of generations) ``` $ nix-env --delete-generations +5 ``` Suppose `30` is the current generation, and we currently have generations numbered `20` through `32`. Then this command will delete generations `20` through `25` (`<= 30 - 5`), and keep generations `26` through `31` (`> 30 - 5`). ## Keep most-recent by time (number of days) ``` $ nix-env --delete-generations 30d ``` This command will delete all generations older than 30 days, except for the generation that was active 30 days ago (if it currently exists). ## Delete all older ``` $ nix-env --profile other_profile --delete-generations old ``` ## [Environment variables](https://docharvest.github.io/docs/nix/command-ref/nix-env/env-common/) Contents nix Environment variables Nix Environment variables - `NIX_PROFILE` Location of the Nix profile. Defaults to the target of the symlink `~/.nix-profile`, if it exists, or `/nix/var/nix/profiles/default` otherwise. ## [Install](https://docharvest.github.io/docs/nix/command-ref/nix-env/install/) Contents nix Install Nix Install `nix-env --install` - add packages to user environment # Synopsis `nix-env` {`--install` | `-i`} _args…_ \[{`--prebuilt-only` | `-b`}\] \[{`--attr` | `-A`}\] \[`--from-expression`\] \[`-E`\] \[`--from-profile` _path_\] \[`--preserve-installed` | `-P`\] \[`--remove-all` | `-r`\] \[`--priority` _priority_\] # Description The `--install` operation creates a new user environment. It is based on the current generation of the active [profile](@docroot@/command-ref/files/profiles.md), to which a set of [store paths](@docroot@/store/store-path.md) described by _args_ is added. The arguments _args_ map to store paths in a number of possible ways: - By default, _args_ is a set of names denoting derivations in the [default Nix expression](@docroot@/command-ref/files/default-nix-expression.md). These are [realised](@docroot@/glossary.md#gloss-realise), and the resulting output paths are installed. Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified. If there are multiple derivations matching a name in _args_ that have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the derivation with the highest _priority_ is used. A derivation can define a priority by declaring the `meta.priority` attribute. This attribute should be a number, with a higher value denoting a lower priority. The default priority is `5`. If there are multiple matching derivations with the same priority, then the derivation with the highest version will be installed. You can force the installation of multiple derivations with the same name by being specific about the versions. For instance, `nix-env --install gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will probably cause a user environment conflict!). - If [`--attr`](#opt-attr) / `-A` is specified, the arguments are _attribute paths_ that select attributes from the [default Nix expression](@docroot@/command-ref/files/default-nix-expression.md). This is faster than using derivation names and unambiguous. Show the attribute paths of available packages with [`nix-env --query`](./query.md): ``` nix-env --query --available --attr-path ``` - If `--from-profile` _path_ is given, _args_ is a set of names denoting installed [store paths](@docroot@/store/store-path.md) in the profile _path_. This is an easy way to copy user environment elements from one profile to another. - If `--from-expression` is given, _args_ are [Nix language functions](@docroot@/language/syntax.md#functions) that are called with the [default Nix expression](@docroot@/command-ref/files/default-nix-expression.md) as their single argument. The derivations returned by those function calls are installed. This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name. - If `--priority` _priority_ is given, the priority of the derivations being installed is set to _priority_. This can be used to override the priority of the derivations being installed. This is useful if _args_ are [store paths](@docroot@/store/store-path.md), which don't have any priority information. - If _args_ are [store paths](@docroot@/store/store-path.md) that point to [store derivations](@docroot@/glossary.md#gloss-store-derivation), then those store derivations are [realised](@docroot@/glossary.md#gloss-realise), and the resulting output paths are installed. - If _args_ are [store paths](@docroot@/store/store-path.md) that do not point to store derivations, then these are [realised](@docroot@/glossary.md#gloss-realise) and installed. - By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [store derivation](@docroot@/glossary.md#gloss-store-derivation). This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names. Example: The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file. ``` # example.nix let pkgs = import {}; command = '' ${pkgs.coreutils}/bin/mkdir -p $foo $bar echo foo > $foo/foo-file echo bar > $bar/bar-file ''; in derivation { name = "example"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" command ]; outputs = [ "foo" "bar" ]; system = builtins.currentSystem; } ``` Installing from this Nix expression will make files from both outputs appear in the current profile. ``` $ nix-env --install --file example.nix installing 'example' $ ls ~/.nix-profile foo-file bar-file manifest.nix ``` Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs. ``` # example-outputs.nix import ./example.nix // { meta.outputsToInstall = [ "bar" ]; } ``` ``` $ nix-env --install --file example-outputs.nix installing 'example' $ ls ~/.nix-profile bar-file manifest.nix ``` # Options - `--prebuilt-only` / `-b` Use only derivations for which a substitute is registered, i.e., there is a pre-built binary available that can be downloaded in lieu of building the derivation. Thus, no packages will be built from source. - `--preserve-installed` / `-P` Do not remove derivations with a name matching one of the derivations being installed. Usually, trying to have two versions of the same package installed in the same generation of a profile will lead to an error in building the generation, due to file name clashes between the two versions. However, this is not the case for all packages. - `--remove-all` / `-r` Remove all previously installed packages first. This is equivalent to running `nix-env --uninstall '.*'` first, except that everything happens in a single transaction. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples To install a package using a specific attribute path from the active Nix expression: ``` $ nix-env --install --attr gcc40mips installing `gcc-4.0.2' $ nix-env --install --attr xorg.xorgserver installing `xorg-server-1.2.0' ``` To install a specific version of `gcc` using the derivation name: ``` $ nix-env --install gcc-3.3.2 installing `gcc-3.3.2' uninstalling `gcc-3.1' ``` Using attribute path for selecting a package is preferred, as it is much faster and there will not be multiple matches. Note the previously installed version is removed, since `--preserve-installed` was not specified. To install an arbitrary version: ``` $ nix-env --install gcc installing `gcc-3.3.2' ``` To install all derivations in the Nix expression `foo.nix`: ``` $ nix-env --file ~/foo.nix --install '.*' ``` To copy the store path with symbolic name `gcc` from another profile: ``` $ nix-env --install --from-profile /nix/var/nix/profiles/foo gcc ``` To install a specific [store derivation](@docroot@/glossary.md#gloss-store-derivation) (typically created by `nix-instantiate`): ``` $ nix-env --install /nix/store/8la6y31fmm6i4wfmby6avly1wf718xnj-gcc-3.4.3.drv ``` To install a specific output path: ``` $ nix-env --install /nix/store/y3cgx0xj1p4iv9x0pnnmdhr8iyg741vk-gcc-3.4.3 ``` To install from a Nix expression specified on the command-line: ``` $ nix-env --file ./foo.nix --install --expr \ 'f: (f {system = "i686-linux";}).subversionWithJava' ``` I.e., this evaluates to `(f: (f {system = "i686-linux";}).subversionWithJava) (import ./foo.nix)`, thus selecting the `subversionWithJava` attribute from the set returned by calling the function defined in `./foo.nix`. A dry-run tells you which paths will be downloaded or built from source: ``` $ nix-env --file '' --install --attr hello --dry-run (dry run; not doing anything) installing ‘hello-2.10’ this path will be fetched (0.04 MiB download, 0.19 MiB unpacked): /nix/store/ikwkxz4wwlp2g1428n7dy729cg1d9hin-hello-2.10 ... ``` To install Firefox from the latest revision in the Nixpkgs/NixOS 14.12 channel: ``` $ nix-env --file https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz --install --attr firefox ``` ## [List Generations](https://docharvest.github.io/docs/nix/command-ref/nix-env/list-generations/) Contents nix List Generations Nix List Generations `nix-env --list-generations` - list profile generations # Synopsis `nix-env` `--list-generations` # Description This operation print a list of all the currently existing generations for the active profile. These may be switched to using the `--switch-generation` operation. It also prints the creation date of the generation, and indicates the current generation. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --list-generations 95 2004-02-06 11:48:24 96 2004-02-06 11:49:01 97 2004-02-06 16:22:45 98 2004-02-06 16:24:33 (current) ``` ## [Opt Common](https://docharvest.github.io/docs/nix/command-ref/nix-env/opt-common/) Contents nix Opt Common Nix Opt Common The following options are allowed for all `nix-env` operations, but may not always have an effect. - `--file` / `-f` _path_ Specifies the Nix expression (designated below as the _active Nix expression_) used by the `--install`, `--upgrade`, and `--query --available` operations to obtain derivations. The default is `~/.nix-defexpr`. If the argument starts with `http://` or `https://`, it is interpreted as the URL of a tarball that will be downloaded and unpacked to a temporary location. The tarball must include a single top-level directory containing at least a file named `default.nix`. - `--profile` / `-p` _path_ Specifies the profile to be used by those operations that operate on a profile (designated below as the _active profile_). A profile is a sequence of user environments called _generations_, one of which is the _current generation_. - `--dry-run` For the `--install`, `--upgrade`, `--uninstall`, `--switch-generation`, `--delete-generations` and `--rollback` operations, this flag will cause `nix-env` to print what _would_ be done if this flag had not been specified, without actually doing it. `--dry-run` also prints out which paths will be [substituted](@docroot@/glossary.md) (i.e., downloaded) and which paths will be built from source (because no substitute is available). - `--system-filter` _system_ By default, operations such as `--query --available` show derivations matching any platform. This option allows you to use derivations for the specified platform _system_. ## [Query](https://docharvest.github.io/docs/nix/command-ref/nix-env/query/) Contents nix Query Nix Query `nix-env --query` - display information about packages # Synopsis `nix-env` {`--query` | `-q`} _names…_ \[`--installed` | `--available` | `-a`\] \[{`--status` | `-s`}\] \[{`--attr-path` | `-P`}\] \[`--no-name`\] \[{`--compare-versions` | `-c`}\] \[`--system`\] \[`--drv-path`\] \[`--out-path`\] \[`--description`\] \[`--meta`\] \[`--xml`\] \[`--json`\] \[{`--prebuilt-only` | `-b`}\] \[{`--attr` | `-A`} _attribute-path_\] # Description The query operation displays information about either the store paths that are installed in the current generation of the active profile (`--installed`), or the derivations that are available for installation in the active Nix expression (`--available`). It only prints information about derivations whose symbolic name matches one of _names_. The derivations are sorted by their `name` attributes. # Source selection The following flags specify the set of things on which the query operates. - `--installed` The query operates on the store paths that are installed in the current generation of the active profile. This is the default. - `--available` / `-a` The query operates on the derivations that are available in the active Nix expression. # Queries The following flags specify what information to display about the selected derivations. Multiple flags may be specified, in which case the information is shown in the order given here. Note that the name of the derivation is shown unless `--no-name` is specified. - `--xml` Print the result in an XML representation suitable for automatic processing by other tools. The root element is called `items`, which contains a `item` element for each available or installed derivation. The fields discussed below are all stored in attributes of the `item` elements. - `--json` Print the result in a JSON representation suitable for automatic processing by other tools. - `--prebuilt-only` / `-b` Show only derivations for which a substitute is registered, i.e., there is a pre-built binary available that can be downloaded in lieu of building the derivation. Thus, this shows all packages that probably can be installed quickly. - `--status` / `-s` Print the _status_ of the derivation. The status consists of three characters. The first is `I` or `-`, indicating whether the derivation is currently installed in the current generation of the active profile. This is by definition the case for `--installed`, but not for `--available`. The second is `P` or `-`, indicating whether the derivation is present on the system. This indicates whether installation of an available derivation will require the derivation to be built. The third is `S` or `-`, indicating whether a substitute is available for the derivation. - `--attr-path` / `-P` Print the _attribute path_ of the derivation, which can be used to unambiguously select it using the `--attr` option available in commands that install derivations like `nix-env --install`. This option only works together with `--available` - `--no-name` Suppress printing of the `name` attribute of each derivation. - `--compare-versions` / `-c` Compare installed versions to available versions, or vice versa (if `--available` is given). This is useful for quickly seeing whether upgrades for installed packages are available in a Nix expression. A column is added with the following meaning: - `<` _version_ A newer version of the package is available or installed. - `=` _version_ At most the same version of the package is available or installed. - `>` _version_ Only older versions of the package are available or installed. - `- ?` No version of the package is available or installed. - `--system` Print the `system` attribute of the derivation. - `--drv-path` Print the [store path](@docroot@/glossary.md#gloss-store-path) to the [store derivation](@docroot@/glossary.md#gloss-derivation). - `--out-path` Print the output path of the derivation. - `--description` Print a short (one-line) description of the derivation, if available. The description is taken from the `meta.description` attribute of the derivation. - `--meta` Print all of the meta-attributes of the derivation. This option is only available with `--xml` or `--json`. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples To show installed packages: ``` $ nix-env --query bison-1.875c docbook-xml-4.2 firefox-1.0.4 MPlayer-1.0pre7 ORBit2-2.8.3 … ``` To show available packages: ``` $ nix-env --query --available firefox-1.0.7 GConf-2.4.0.1 MPlayer-1.0pre7 ORBit2-2.8.3 … ``` To show the status of available packages: ``` $ nix-env --query --available --status -P- firefox-1.0.7 (not installed but present) --S GConf-2.4.0.1 (not present, but there is a substitute for fast installation) --S MPlayer-1.0pre3 (i.e., this is not the installed MPlayer, even though the version is the same!) IP- ORBit2-2.8.3 (installed and by definition present) … ``` To show available packages in the Nix expression `foo.nix`: ``` $ nix-env --file ./foo.nix --query --available foo-1.2.3 ``` To compare installed versions to what’s available: ``` $ nix-env --query --compare-versions ... acrobat-reader-7.0 - ? (package is not available at all) autoconf-2.59 = 2.59 (same version) firefox-1.0.4 < 1.0.7 (a more recent version is available) ... ``` To show all packages with “`zip`” in the name: ``` $ nix-env --query --available '.*zip.*' bzip2-1.0.6 gzip-1.6 zip-3.0 … ``` To show all packages with “`firefox`” or “`chromium`” in the name: ``` $ nix-env --query --available '.*(firefox|chromium).*' chromium-37.0.2062.94 chromium-beta-38.0.2125.24 firefox-32.0.3 firefox-with-plugins-13.0.1 … ``` To show all packages in the latest revision of the Nixpkgs repository: ``` $ nix-env --file https://github.com/NixOS/nixpkgs/archive/master.tar.gz --query --available ``` ## [Rollback](https://docharvest.github.io/docs/nix/command-ref/nix-env/rollback/) Contents nix Rollback Nix Rollback `nix-env --rollback` - set user environment to previous generation # Synopsis `nix-env` `--rollback` # Description This operation switches to the “previous” generation of the active profile, that is, the highest numbered generation lower than the current generation, if it exists. It is just a convenience wrapper around `--list-generations` and `--switch-generation`. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --rollback switching from generation 92 to 91 ``` ``` $ nix-env --rollback error: no generation older than the current (91) exists ``` ## [Name](https://docharvest.github.io/docs/nix/command-ref/nix-env/set/) Contents nix Name Nix Name `nix-env --set` - set profile to contain a specified derivation ## Synopsis `nix-env` `--set` _drvname_ ## Description The `--set` operation modifies the current generation of a profile so that it contains exactly the specified derivation, and nothing else. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} ## Examples The following updates a profile such that its current generation will contain just Firefox: ``` $ nix-env --profile /nix/var/nix/profiles/browser --set firefox ``` ## [Set Flag](https://docharvest.github.io/docs/nix/command-ref/nix-env/set-flag/) Contents nix Set Flag Nix Set Flag `nix-env --set-flag` - modify meta attributes of installed packages # Synopsis `nix-env` `--set-flag` _name_ _value_ _drvnames_ # Description The `--set-flag` operation allows meta attributes of installed packages to be modified. There are several attributes that can be usefully modified, because they affect the behaviour of `nix-env` or the user environment build script: - `priority` can be changed to resolve filename clashes. The user environment build script uses the `meta.priority` attribute of derivations to resolve filename collisions between packages. Lower priority values denote a higher priority. For instance, the GCC wrapper package and the Binutils package in Nixpkgs both have a file `bin/ld`, so previously if you tried to install both you would get a collision. Now, on the other hand, the GCC wrapper declares a higher priority than Binutils, so the former’s `bin/ld` is symlinked in the user environment. - `keep` can be set to `true` to prevent the package from being upgraded or replaced. This is useful if you want to hang on to an older version of a package. - `active` can be set to `false` to “disable” the package. That is, no symlinks will be generated to the files of the package, but it remains part of the profile (so it won’t be garbage-collected). It can be set back to `true` to re-enable the package. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples To prevent the currently installed Firefox from being upgraded: ``` $ nix-env --set-flag keep true firefox ``` After this, `nix-env --upgrade` will ignore Firefox. To disable the currently installed Firefox, then install a new Firefox while the old remains part of the profile: ``` $ nix-env --query firefox-2.0.0.9 (the current one) $ nix-env --preserve-installed --install firefox-2.0.0.11 installing `firefox-2.0.0.11' building path(s) `/nix/store/myy0y59q3ig70dgq37jqwg1j0rsapzsl-user-environment' collision between `/nix/store/...-firefox-2.0.0.11/bin/firefox' and `/nix/store/...-firefox-2.0.0.9/bin/firefox'. (i.e., can’t have two active at the same time) $ nix-env --set-flag active false firefox setting flag on `firefox-2.0.0.9' $ nix-env --preserve-installed --install firefox-2.0.0.11 installing `firefox-2.0.0.11' $ nix-env --query firefox-2.0.0.11 (the enabled one) firefox-2.0.0.9 (the disabled one) ``` To make files from `binutils` take precedence over files from `gcc`: ``` $ nix-env --set-flag priority 5 binutils $ nix-env --set-flag priority 10 gcc ``` ## [Switch Generation](https://docharvest.github.io/docs/nix/command-ref/nix-env/switch-generation/) Contents nix Switch Generation Nix Switch Generation `nix-env --switch-generation` - set user environment to given profile generation # Synopsis `nix-env` {`--switch-generation` | `-G`} _generation_ # Description This operation makes generation number _generation_ the current generation of the active profile. That is, if the `profile` is the path to the active profile, then the symlink `profile` is made to point to `profile-generation-link`, which is in turn a symlink to the actual user environment in the Nix store. Switching will fail if the specified generation does not exist. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --switch-generation 42 switching from generation 50 to 42 ``` ## [Switch Profile](https://docharvest.github.io/docs/nix/command-ref/nix-env/switch-profile/) Contents nix Switch Profile Nix Switch Profile `nix-env --switch-profile` - set user environment to given profile # Synopsis `nix-env` {`--switch-profile` | `-S`} _path_ # Description This operation makes _path_ the current profile for the user. That is, the symlink `~/.nix-profile` is made to point to _path_. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --switch-profile ~/my-profile ``` ## [Uninstall](https://docharvest.github.io/docs/nix/command-ref/nix-env/uninstall/) Contents nix Uninstall Nix Uninstall `nix-env --uninstall` - remove packages from user environment # Synopsis `nix-env` {`--uninstall` | `-e`} _drvnames…_ # Description The uninstall operation creates a new user environment, based on the current generation of the active profile, from which the store paths designated by the symbolic names _drvnames_ are removed. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --uninstall gcc $ nix-env --uninstall '.*' (remove everything) ``` ## [Upgrade](https://docharvest.github.io/docs/nix/command-ref/nix-env/upgrade/) Contents nix Upgrade Nix Upgrade `nix-env --upgrade` - upgrade packages in user environment # Synopsis `nix-env` {`--upgrade` | `-u`} _args_ \[`--lt` | `--leq` | `--eq` | `--always`\] \[{`--prebuilt-only` | `-b`}\] \[{`--attr` | `-A`}\] \[`--from-expression`\] \[`-E`\] \[`--from-profile` _path_\] \[`--preserve-installed` | `-P`\] # Description The upgrade operation creates a new user environment, based on the current generation of the active profile, in which all store paths are replaced for which there are newer versions in the set of paths described by _args_. Paths for which there are no newer versions are left untouched; this is not an error. It is also not an error if an element of _args_ matches no installed derivations. For a description of how _args_ is mapped to a set of store paths, see [`--install`](./install.md). If _args_ describes multiple store paths with the same symbolic name, only the one with the highest version is installed. # Flags - `--lt` Only upgrade a derivation to newer versions. This is the default. - `--leq` In addition to upgrading to newer versions, also “upgrade” to derivations that have the same version. Version are not a unique identification of a derivation, so there may be many derivations that have the same version. This flag may be useful to force “synchronisation” between the installed and available derivations. - `--eq` _Only_ “upgrade” to derivations that have the same version. This may not seem very useful, but it actually is, e.g., when there is a new release of Nixpkgs and you want to replace installed applications with the same versions built against newer dependencies (to reduce the number of dependencies floating around on your system). - `--always` In addition to upgrading to newer versions, also “upgrade” to derivations that have the same or a lower version. I.e., derivations may actually be downgraded depending on what is available in the active Nix expression. - `--prebuilt-only` / `-b` Use only derivations for which a substitute is registered, i.e., there is a pre-built binary available that can be downloaded in lieu of building the derivation. Thus, no packages will be built from source. - `--preserve-installed` / `-P` Do not remove derivations with a name matching one of the derivations being installed. Usually, trying to have two versions of the same package installed in the same generation of a profile will lead to an error in building the generation, due to file name clashes between the two versions. However, this is not the case for all packages. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ./env-common.md}} {{#include ../env-common.md}} # Examples ``` $ nix-env --upgrade --attr nixpkgs.gcc upgrading `gcc-3.3.1' to `gcc-3.4' ``` When there are no updates available, nothing will happen: ``` $ nix-env --upgrade --attr nixpkgs.pan ``` Using `-A` is preferred when possible, as it is faster and unambiguous but it is also possible to upgrade to a specific version by matching the derivation name: ``` $ nix-env --upgrade gcc-3.3.2 --always upgrading `gcc-3.4' to `gcc-3.3.2' ``` To try to upgrade everything (matching packages based on the part of the derivation name without version): ``` $ nix-env --upgrade upgrading `hello-2.1.2' to `hello-2.1.3' upgrading `mozilla-1.2' to `mozilla-1.4' ``` # Versions The upgrade operation determines whether a derivation `y` is an upgrade of a derivation `x` by looking at their respective `name` attributes. The names (e.g., `gcc-3.3.1` are split into two parts: the package name (`gcc`), and the version (`3.3.1`). The version part starts after the first dash not followed by a letter. `y` is considered an upgrade of `x` if their package names match, and the version of `y` is higher than that of `x`. The versions are compared by splitting them into contiguous components of numbers and letters. E.g., `3.3.1pre5` is split into `[3, 3, 1, "pre", 5]`. These lists are then compared lexicographically (from left to right). Corresponding components `a` and `b` are compared as follows. If they are both numbers, integer comparison is used. If `a` is an empty string and `b` is a number, `a` is considered less than `b`. The special string component `pre` (for _pre-release_) is considered to be less than other components. String components are considered less than number components. Otherwise, they are compared lexicographically (i.e., using case-sensitive string comparison). This is illustrated by the following examples: ``` 1.0 < 2.3 2.1 < 2.3 2.3 = 2.3 2.5 > 2.3 3.1 > 2.3 2.3.1 > 2.3 2.3.1 > 2.3a 2.3pre1 < 2.3 2.3pre3 < 2.3pre12 2.3a < 2.3c 2.3pre1 < 2.3c 2.3pre1 < 2.3q ``` ## [Nix Hash](https://docharvest.github.io/docs/nix/command-ref/nix-hash/) Contents nix Nix Hash Nix Nix Hash `nix-hash` - compute the cryptographic hash of a path # Synopsis `nix-hash` \[`--flat`\] \[`--base32`\] \[`--truncate`\] \[`--type` _hashAlgo_\] _path…_ `nix-hash` \[`--to-base16`|`--to-base32`|`--to-base64`|`--to-sri`\] \[`--type` _hashAlgo_\] _hash…_ # Description The command `nix-hash` computes the cryptographic hash of the contents of each _path_ and prints it on standard output. By default, it computes an MD5 hash, but other hash algorithms are available as well. The hash is printed in hexadecimal. To generate the same hash as `nix-prefetch-url` you have to specify multiple arguments, see below for an example. The hash is computed over a _serialisation_ of each path: a dump of the file system tree rooted at the path. This allows directories and symlinks to be hashed as well as regular files. The dump is in the _[Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) format_ produced by [`nix-store --dump`](@docroot@/command-ref/nix-store/dump.md). Thus, `nix-hash path` yields the same cryptographic hash as `nix-store --dump path | md5sum`. # Options - `--flat` Print the cryptographic hash of the contents of each regular file _path_. That is, instead of computing the hash of the [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) of _path_, just [directly hash](@docroot@/store/file-system-object/content-address.md#serial-flat) _path_ as is. This requires _path_ to resolve to a regular file rather than directory. The result is identical to that produced by the GNU commands `md5sum` and `sha1sum`. - `--base16` Print the hash in a hexadecimal representation (default). - `--base32` Print the hash in [Nix32](@docroot@/protocols/nix32.md) representation rather than hexadecimal. This representation is more compact and can be used in Nix expressions (such as in calls to `fetchurl`). - `--base64` Similar to `--base32`, but print the hash in a [Base64](https://en.wikipedia.org/wiki/Base64) representation, which is more compact than the Nix32 one. - `--sri` Print the hash in [SRI](@docroot@/glossary.md#gloss-sri) format with Base64 encoding. The type of hash algorithm will be prepended to the hash string, followed by a hyphen (-) and the Base64 hash body. - `--truncate` Truncate hashes longer than 160 bits (such as SHA-256) to 160 bits. - `--type` _hashAlgo_ Use the specified cryptographic hash algorithm, which can be one of `blake3`, `md5`, `sha1`, `sha256`, and `sha512`. - `--to-base16` Don’t hash anything, but convert the [Nix32](@docroot@/protocols/nix32.md) hash representation _hash_ to hexadecimal. - `--to-base32` Don’t hash anything, but convert the hexadecimal hash representation _hash_ to [Nix32](@docroot@/protocols/nix32.md). - `--to-base64` Don’t hash anything, but convert the hexadecimal hash representation _hash_ to Base64. - `--to-sri` Don’t hash anything, but convert the hexadecimal hash representation _hash_ to SRI. # Examples Computing the same hash as `nix-prefetch-url`: ``` $ nix-prefetch-url file://<(echo test) 1lkgqb6fclns49861dwk9rzb6xnfkxbpws74mxnx01z9qyv1pjpj $ nix-hash --type sha256 --flat --base32 <(echo test) 1lkgqb6fclns49861dwk9rzb6xnfkxbpws74mxnx01z9qyv1pjpj ``` Computing hashes: ``` $ mkdir test $ echo "hello" > test/world $ nix-hash test/ (MD5 hash; default) 8179d3caeff1869b5ba1744e5a245c04 $ nix-store --dump test/ | md5sum (for comparison) 8179d3caeff1869b5ba1744e5a245c04 - $ nix-hash --type sha1 test/ e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 $ nix-hash --type sha1 --base16 test/ e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 $ nix-hash --type sha1 --base32 test/ nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 $ nix-hash --type sha1 --base64 test/ 5P2Lpfe76upazon+ECVVNs1g2rY= $ nix-hash --type sha1 --sri test/ sha1-5P2Lpfe76upazon+ECVVNs1g2rY= $ nix-hash --type sha256 --flat test/ error: reading file `test/': Is a directory $ nix-hash --type sha256 --flat test/world 5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03 ``` Converting between hexadecimal, Nix32, Base64, and SRI: ``` $ nix-hash --type sha1 --to-base32 e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 $ nix-hash --type sha1 --to-base16 nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 $ nix-hash --type sha1 --to-base64 e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 5P2Lpfe76upazon+ECVVNs1g2rY= $ nix-hash --type sha1 --to-sri nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 sha1-5P2Lpfe76upazon+ECVVNs1g2rY= $ nix-hash --to-base16 sha1-5P2Lpfe76upazon+ECVVNs1g2rY= e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 ``` ## [Nix Instantiate](https://docharvest.github.io/docs/nix/command-ref/nix-instantiate/) Contents nix Nix Instantiate Nix Nix Instantiate `nix-instantiate` - instantiate store derivations from Nix expressions # Synopsis `nix-instantiate` \[`--parse` | `--eval` \[`--strict`\] \[`--raw` | `--json` | `--xml`\] \] \[`--read-write-mode`\] \[`--arg` _name_ _value_\] \[{`--attr`| `-A`} _attrPath_\] \[`--add-root` _path_\] \[`--expr` | `-E`\] _files…_ `nix-instantiate` `--find-file` _files…_ # Description The command `nix-instantiate` produces [store derivation](@docroot@/glossary.md#gloss-store-derivation)s from (high-level) Nix expressions. It evaluates the Nix expressions in each of _files_ (which defaults to _./default.nix_). Each top-level expression should evaluate to a derivation, a list of derivations, or a set of derivations. The paths of the resulting store derivations are printed on standard output. If _files_ is the character `-`, then a Nix expression will be read from standard input. # Options - `--add-root` _path_ See the [corresponding option](./nix-store.md) in `nix-store`. - `--parse` Just parse the input files, and print their abstract syntax trees on standard output as a Nix expression. - `--eval` Just parse and evaluate the input files, and print the resulting values on standard output. Store derivations are not serialized and written to the store, but instead just hashed and discarded. > **Warning** > > This option produces output which can be parsed as a Nix expression which will produce a different result than the input expression when evaluated. For example, these two Nix expressions print the same result despite having different meaning: > > ``` > $ nix-instantiate --eval --expr '{ a = {}; }' > { a = ; } > $ nix-instantiate --eval --expr '{ a = ; }' > { a = ; } > ``` > > For human-readable output, `nix eval` (experimental) is more informative: > > ``` > $ nix-instantiate --eval --expr 'a: a' > > $ nix eval --expr 'a: a' > «lambda @ «string»:1:1» > ``` > > For machine-readable output, the `--xml` option produces unambiguous output: > > ``` > $ nix-instantiate --eval --xml --expr '{ foo = ; }' > > > > > > > > > ``` - `--find-file` Look up the given files in Nix’s search path (as specified by the `NIX_PATH` environment variable). If found, print the corresponding absolute paths on standard output. For instance, if `NIX_PATH` is `nixpkgs=/home/alice/nixpkgs`, then `nix-instantiate --find-file nixpkgs/default.nix` will print `/home/alice/nixpkgs/default.nix`. - `--strict` When used with `--eval`, recursively evaluate list elements and attributes. Normally, such sub-expressions are left unevaluated (since the Nix language is lazy). > **Warning** > > This option can cause non-termination, because lazy data structures can be infinitely large. - `--raw` When used with `--eval`, the evaluation result must be a string, which is printed verbatim, without quoting, escaping or trailing newline. - `--json` When used with `--eval`, print the resulting value as an JSON representation of the abstract syntax tree rather than as a Nix expression. - `--xml` When used with `--eval`, print the resulting value as an XML representation of the abstract syntax tree rather than as a Nix expression. The schema is the same as that used by the [`toXML` built-in](../language/builtins.md). - `--read-write-mode` When used with `--eval`, perform evaluation in read/write mode so nix language features that require it will still work (at the cost of needing to do instantiation of every evaluated derivation). If this option is not enabled, there may be uninstantiated store paths in the final output. {{#include ./opt-common.md}} {{#include ./env-common.md}} # Examples Instantiate [store derivation](@docroot@/glossary.md#gloss-store-derivation)s from a Nix expression, and build them using `nix-store`: ``` $ nix-instantiate test.nix (instantiate) /nix/store/cigxbmvy6dzix98dxxh9b6shg7ar5bvs-perl-BerkeleyDB-0.26.drv $ nix-store --realise $(nix-instantiate test.nix) (build) ... /nix/store/qhqk4n8ci095g3sdp93x7rgwyh9rdvgk-perl-BerkeleyDB-0.26 (output path) $ ls -l /nix/store/qhqk4n8ci095g3sdp93x7rgwyh9rdvgk-perl-BerkeleyDB-0.26 dr-xr-xr-x 2 eelco users 4096 1970-01-01 01:00 lib ... ``` You can also give a Nix expression on the command line: ``` $ nix-instantiate --expr 'with import { }; hello' /nix/store/j8s4zyv75a724q38cb0r87rlczaiag4y-hello-2.8.drv ``` This is equivalent to: ``` $ nix-instantiate '' --attr hello ``` Parsing and evaluating Nix expressions: ``` $ nix-instantiate --parse --expr '1 + 2' 1 + 2 ``` ``` $ nix-instantiate --eval --expr '1 + 2' 3 ``` ``` $ nix-instantiate --eval --xml --expr '1 + 2' ``` The difference between non-strict and strict evaluation: ``` $ nix-instantiate --eval --xml --expr '{ x = {}; }' $ nix-instantiate --eval --xml --strict --expr '{ x = {}; }' ``` ## [Nix Prefetch Url](https://docharvest.github.io/docs/nix/command-ref/nix-prefetch-url/) Contents nix Nix Prefetch Url Nix Nix Prefetch Url `nix-prefetch-url` - copy a file from a URL into the store and print its hash # Synopsis `nix-prefetch-url` _url_ \[_hash_\] \[`--type` _hashAlgo_\] \[`--print-path`\] \[`--unpack`\] \[`--name` _name_\] # Description The command `nix-prefetch-url` downloads the file referenced by the URL _url_, prints its cryptographic hash, and copies it into the Nix store. The file name in the store is `hash-baseName`, where _baseName_ is everything following the final slash in _url_. This command is just a convenience for Nix expression writers. Often a Nix expression fetches some source distribution from the network using the `fetchurl` expression contained in Nixpkgs. However, `fetchurl` requires a cryptographic hash. If you don't know the hash, you would have to download the file first, and then `fetchurl` would download it again when you build your Nix expression. Since `fetchurl` uses the same name for the downloaded file as `nix-prefetch-url`, the redundant download can be avoided. If _hash_ is specified, then a download is not performed if the Nix store already contains a file with the same hash and base name. Otherwise, the file is downloaded, and an error is signaled if the actual hash of the file does not match the specified hash. This command prints the hash on standard output. The hash is printed using [Nix32](@docroot@/protocols/nix32.md) unless `--type md5` is specified, in which case it's printed using base-16. Additionally, if the option `--print-path` is used, the path of the downloaded file in the Nix store is also printed. # Options - `--type` _hashAlgo_ Use the specified cryptographic hash algorithm, which can be one of `blake3`, `md5`, `sha1`, `sha256`, and `sha512`. The default is `sha256`. - `--print-path` Print the store path of the downloaded file on standard output. - `--unpack` Unpack the archive (which must be a tarball or zip file) and add the result to the Nix store. The resulting hash can be used with functions such as Nixpkgs’s `fetchzip` or `fetchFromGitHub`. - `--executable` Set the executable bit on the downloaded file. - `--name` _name_ Override the name of the file in the Nix store. By default, this is `hash-basename`, where _basename_ is the last component of _url_. Overriding the name is necessary when _basename_ contains characters that are not allowed in Nix store paths. # Examples ``` $ nix-prefetch-url ftp://ftp.gnu.org/pub/gnu/hello/hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i ``` ``` $ nix-prefetch-url --print-path mirror://gnu/hello/hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i /nix/store/8alrpdaasjd1x6g1fczchmzbpqm936a3-hello-2.10.tar.gz ``` ``` $ nix-prefetch-url --unpack --print-path https://github.com/NixOS/patchelf/archive/0.8.tar.gz 079agjlv0hrv7fxnx9ngipx14gyncbkllxrp9cccnh3a50fxcmy7 /nix/store/19zrmhm3m40xxaw81c8cqm6aljgrnwj2-0.8.tar.gz ``` ## [Nix Shell](https://docharvest.github.io/docs/nix/command-ref/nix-shell/) Contents nix Nix Shell Nix Nix Shell `nix-shell` - start an interactive shell based on a Nix expression # Synopsis `nix-shell` \[`--arg` _name_ _value_\] \[`--argstr` _name_ _value_\] \[{`--attr` | `-A`} _attrPath_\] \[`--command` _cmd_\] \[`--run` _cmd_\] \[`--exclude` _regexp_\] \[`--pure`\] \[`--keep` _name_\] {{`--packages` | `-p`} {_packages_ | _expressions_} … | \[_path_\]} # Disambiguation This man page describes the command `nix-shell`, which is distinct from `nix shell`. For documentation on the latter, run `nix shell --help` or see `man nix3-env-shell`. # Description The command `nix-shell` will build the dependencies of the specified derivation, but not the derivation itself. It will then start an interactive shell in which all environment variables defined by the derivation _path_ have been set to their corresponding values, and the script `$stdenv/setup` has been sourced. This is useful for reproducing the environment of a derivation for development. If _path_ is not given, `nix-shell` defaults to `shell.nix` if it exists, and `default.nix` otherwise. If _path_ starts with `http://` or `https://`, it is interpreted as the URL of a tarball that will be downloaded and unpacked to a temporary location. The tarball must include a single top-level directory containing at least a file named `default.nix`. If the derivation defines the variable `shellHook`, it will be run after `$stdenv/setup` has been sourced. Since this hook is not executed by regular Nix builds, it allows you to perform initialisation specific to `nix-shell`. For example, the derivation attribute ``` shellHook = '' echo "Hello shell" export SOME_API_TOKEN="$(cat ~/.config/some-app/api-token)" ''; ``` will cause `nix-shell` to print `Hello shell` and set the `SOME_API_TOKEN` environment variable to a user-configured value. # Options All options not listed here are passed to `nix-store --realise`, except for `--arg` and `--attr` / `-A` which are passed to `nix-instantiate`. - `--command` _cmd_ In the environment of the derivation, run the shell command _cmd_. This command is executed in an interactive shell. (Use `--run` to use a non-interactive shell instead.) However, a call to `exit` is implicitly added to the command, so the shell will exit after running the command. To prevent this, add `return` at the end; e.g. `--command "echo Hello; return"` will print `Hello` and then drop you into the interactive shell. This can be useful for doing any additional initialisation. - `--run` _cmd_ Like `--command`, but executes the command in a non-interactive shell. This means (among other things) that if you hit Ctrl-C while the command is running, the shell exits. - `--exclude` _regexp_ Do not build any dependencies whose store path matches the regular expression _regexp_. This option may be specified multiple times. - `--pure` If this flag is specified, the environment is almost entirely cleared before the interactive shell is started, so you get an environment that more closely corresponds to the “real” Nix build. A few variables, in particular `HOME`, `USER` and `DISPLAY`, are retained. Note that the shell used to run commands is obtained from [`NIX_BUILD_SHELL`](#env-NIX_BUILD_SHELL) / `` from `NIX_PATH`, and therefore not affected by `--pure`. - `--packages` / `-p` _packages_… Set up an environment in which the specified packages are present. The command line arguments are interpreted as attribute names inside the Nix Packages collection. Thus, `nix-shell --packages libjpeg openjdk` will start a shell in which the packages denoted by the attribute names `libjpeg` and `openjdk` are present. - `-i` _interpreter_ The chained script interpreter to be invoked by `nix-shell`. Only applicable in `#!`\-scripts (described below). - `--keep` _name_ When a `--pure` shell is started, keep the listed environment variables. {{#include ./opt-common.md}} # Environment variables - [`NIX_BUILD_SHELL`](#env-NIX_BUILD_SHELL) Shell used to start the interactive environment. Defaults to the `bash` from `bashInteractive` found in ``, falling back to the `bash` found in `PATH` if not found. > **Note** > > The shell obtained using this method may not necessarily be the same as any shells requested in _path_. > \*\*Example > > Despite `--pure`, this invocation will not result in a fully reproducible shell environment: > > ``` > #!/usr/bin/env -S nix-shell --pure > let > pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/854fdc68881791812eddd33b2fed94b954979a8e.tar.gz") {}; > in > pkgs.mkShell { > buildInputs = pkgs.bashInteractive; > } > ``` {{#include ./env-common.md}} # Examples To build the dependencies of the package Pan, and start an interactive shell in which to build it: ``` $ nix-shell '' --attr pan [nix-shell]$ eval ${unpackPhase:-unpackPhase} [nix-shell]$ cd $sourceRoot [nix-shell]$ eval ${patchPhase:-patchPhase} [nix-shell]$ eval ${configurePhase:-configurePhase} [nix-shell]$ eval ${buildPhase:-buildPhase} [nix-shell]$ ./pan/gui/pan ``` The reason we use form `eval ${configurePhase:-configurePhase}` here is because those packages that override these phases do so by exporting the overridden values in the environment variable of the same name. Here bash is being told to either evaluate the contents of 'configurePhase', if it exists as a variable, otherwise evaluate the configurePhase function. To clear the environment first, and do some additional automatic initialisation of the interactive shell: ``` $ nix-shell '' --attr pan --pure \ --command 'export NIX_DEBUG=1; export NIX_CORES=8; return' ``` Nix expressions can also be given on the command line using the `-E` and `-p` flags. For instance, the following starts a shell containing the packages `sqlite` and `libX11`: ``` $ nix-shell --expr 'with import { }; runCommand "dummy" { buildInputs = [ sqlite xorg.libX11 ]; } ""' ``` A shorter way to do the same is: ``` $ nix-shell --packages sqlite xorg.libX11 [nix-shell]$ echo $NIX_LDFLAGS … -L/nix/store/j1zg5v…-sqlite-3.8.0.2/lib -L/nix/store/0gmcz9…-libX11-1.6.1/lib … ``` Note that `-p` accepts multiple full nix expressions that are valid in the `buildInputs = [ ... ]` shown above, not only package names. So the following is also legal: ``` $ nix-shell --packages sqlite 'git.override { withManual = false; }' ``` The `-p` flag looks up Nixpkgs in the Nix search path. You can override it by passing `-I` or setting `NIX_PATH`. For example, the following gives you a shell containing the Pan package from a specific revision of Nixpkgs: ``` $ nix-shell --packages pan -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/8a3eea054838b55aca962c3fbde9c83c102b8bf2.tar.gz [nix-shell:~]$ pan --version Pan 0.139 ``` # Use as a `#!`\-interpreter You can use `nix-shell` as a script interpreter to allow scripts written in arbitrary languages to obtain their own dependencies via Nix. This is done by starting the script with the following lines: ``` #! /usr/bin/env nix-shell #! nix-shell -i real-interpreter --packages packages ``` where _real-interpreter_ is the “real” script interpreter that will be invoked by `nix-shell` after it has obtained the dependencies and initialised the environment, and _packages_ are the attribute names of the dependencies in Nixpkgs. The lines starting with `#! nix-shell` specify `nix-shell` options (see above). Note that you cannot write `#! /usr/bin/env nix-shell -i ...` because many operating systems only allow one argument in `#!` lines. For example, here is a Python script that depends on Python and the `prettytable` package: ``` #! /usr/bin/env nix-shell #! nix-shell -i python3 --packages python3 python3Packages.prettytable import prettytable # Print a simple table. t = prettytable.PrettyTable(["N", "N^2"]) for n in range(1, 10): t.add_row([n, n * n]) print(t) ``` Similarly, the following is a Perl script that specifies that it requires Perl and the `HTML::TokeParser::Simple`, `LWP` and `LWP::Protocol::Https` packages: ``` #! /usr/bin/env nix-shell #! nix-shell -i perl #! nix-shell --packages perl #! nix-shell --packages perlPackages.HTMLTokeParserSimple #! nix-shell --packages perlPackages.LWP #! nix-shell --packages perlPackages.LWPProtocolHttps use HTML::TokeParser::Simple; # Fetch nixos.org and print all hrefs. my $p = HTML::TokeParser::Simple->new(url => 'https://nixos.org/'); while (my $token = $p->get_tag("a")) { my $href = $token->get_attr("href"); print "$href\n" if $href; } ``` Sometimes you need to pass a simple Nix expression to customize a package like Terraform: ``` #! /usr/bin/env nix-shell #! nix-shell -i bash --packages 'terraform.withPlugins (plugins: [ plugins.openstack ])' terraform apply ``` > **Note** > > You must use single or double quotes (`'`, `"`) when passing a simple Nix expression in a nix-shell shebang. Finally, using the merging of multiple nix-shell shebangs the following Haskell script uses a specific branch of Nixpkgs/NixOS (the 20.03 stable branch): ``` #! /usr/bin/env nix-shell #! nix-shell -i runghc --packages 'haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])' #! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.03.tar.gz import Network.Curl.Download import Text.HTML.TagSoup import Data.Either import Data.ByteString.Char8 (unpack) -- Fetch nixos.org and print all hrefs. main = do resp <- openURI "https://nixos.org/" let tags = filter (isTagOpenName "a") $ parseTags $ unpack $ fromRight undefined resp let tags' = map (fromAttrib "href") tags mapM_ putStrLn $ filter (/= "") tags' ``` If you want to be even more precise, you can specify a specific revision of Nixpkgs: ``` #! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/0672315759b3e15e2121365f067c1c8c56bb4722.tar.gz ``` The examples above all used `-p` to get dependencies from Nixpkgs. You can also use a Nix expression to build your own dependencies. For example, the Python example could have been written as: ``` #! /usr/bin/env nix-shell #! nix-shell deps.nix -i python ``` where the file `deps.nix` in the same directory as the `#!`\-script contains: ``` with import {}; runCommand "dummy" { buildInputs = [ python3 python3Packages.prettytable ]; } "" ``` The script's file name is passed as the first argument to the interpreter specified by the `-i` flag. Aside from the very first line, which is a directive to the operating system, the additional `#! nix-shell` lines do not need to be at the beginning of the file. This allows wrapping them in block comments for languages where `#` does not start a comment, such as ECMAScript, Erlang, PHP, or Ruby. ## [Nix Store](https://docharvest.github.io/docs/nix/command-ref/nix-store/) Contents nix Nix Store Nix Nix Store `nix-store` - manipulate or query the Nix store # Synopsis `nix-store` _operation_ \[_options…_\] \[_arguments…_\] \[`--option` _name_ _value_\] \[`--add-root` _path_\] # Description The command `nix-store` performs primitive operations on the Nix store. You generally do not need to run this command manually. `nix-store` takes exactly one _operation_ flag which indicates the subcommand to be performed. The following operations are available: - [`--realise`](./nix-store/realise.md) - [`--serve`](./nix-store/serve.md) - [`--gc`](./nix-store/gc.md) - [`--delete`](./nix-store/delete.md) - [`--query`](./nix-store/query.md) - [`--add`](./nix-store/add.md) - [`--add-fixed`](./nix-store/add-fixed.md) - [`--verify`](./nix-store/verify.md) - [`--verify-path`](./nix-store/verify-path.md) - [`--repair-path`](./nix-store/repair-path.md) - [`--dump`](./nix-store/dump.md) - [`--restore`](./nix-store/restore.md) - [`--export`](./nix-store/export.md) - [`--import`](./nix-store/import.md) - [`--optimise`](./nix-store/optimise.md) - [`--read-log`](./nix-store/read-log.md) - [`--dump-db`](./nix-store/dump-db.md) - [`--load-db`](./nix-store/load-db.md) - [`--print-env`](./nix-store/print-env.md) - [`--generate-binary-cache-key`](./nix-store/generate-binary-cache-key.md) These pages can be viewed offline: - `man nix-store-`. Example: `man nix-store-realise` - `nix-store --help --` Example: `nix-store --help --realise` ## [Name](https://docharvest.github.io/docs/nix/command-ref/nix-store/add/) Contents nix Name Nix Name `nix-store --add` - add paths to Nix store # Synopsis `nix-store` `--add` _paths…_ # Description The operation `--add` adds the specified paths to the Nix store. It prints the resulting paths in the Nix store on standard output. _paths_ that refer to symlinks are not dereferenced, but added to the store as symlinks with the same target. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Example ``` $ nix-store --add ./foo.c /nix/store/m7lrha58ph6rcnv109yzx1nk1cj7k7zf-foo.c ``` ## [Add Fixed](https://docharvest.github.io/docs/nix/command-ref/nix-store/add-fixed/) Contents nix Add Fixed Nix Add Fixed `nix-store --add-fixed` - add paths to store using given hashing algorithm ## Synopsis `nix-store` `--add-fixed` \[`--recursive`\] _algorithm_ _paths…_ ## Description The operation `--add-fixed` adds the specified paths to the Nix store. Unlike `--add` paths are registered using the specified hashing algorithm, resulting in the same output path as a fixed-output derivation. This can be used for sources that are not available from a public url or broke since the download expression was written. This operation has the following options: - `--recursive` Use recursive instead of flat hashing mode, used when adding directories to the store. _paths_ that refer to symlinks are not dereferenced, but added to the store as symlinks with the same target. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## Example ``` $ nix-store --add-fixed sha256 ./hello-2.10.tar.gz /nix/store/8alrpdaasjd1x6g1fczchmzbpqm936a3-hello-2.10.tar.gz ``` ## [Delete](https://docharvest.github.io/docs/nix/command-ref/nix-store/delete/) Contents nix Delete Nix Delete `nix-store --delete` - delete store paths # Synopsis `nix-store` `--delete` \[`--ignore-liveness`\] _paths…_ # Description The operation `--delete` deletes the store paths _paths_ from the Nix store, but only if it is safe to do so; that is, when the path is not reachable from a root of the garbage collector. This means that you can only delete paths that would also be deleted by `nix-store --gc`. Thus, `--delete` is a more targeted version of `--gc`. With the option `--ignore-liveness`, reachability from the roots is ignored. However, the path still won’t be deleted if there are other paths in the store that refer to it (i.e., depend on it). {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Example ``` $ nix-store --delete /nix/store/gjak3al7lj61x4gj6rln4f5pc5v0f67n-mesa-6.4 0 bytes freed (0.00 MiB) error: cannot delete path `/nix/store/gjak3al7lj61x4gj6rln4f5pc5v0f67n-mesa-6.4' since it is still alive ``` ## [Name](https://docharvest.github.io/docs/nix/command-ref/nix-store/dump/) Contents nix Name Nix Name `nix-store --dump` - write a single path to a [Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) ## Synopsis `nix-store` `--dump` _path_ ## Description The operation `--dump` produces a [Nix archive](@docroot@/glossary.md#gloss-nar) (NAR) file containing the contents of the file system tree rooted at _path_. The archive is written to standard output. A NAR archive is like a TAR or Zip archive, but it contains only the information that Nix considers important. For instance, timestamps are elided because all files in the Nix store have their timestamp set to 0 anyway. Likewise, all permissions are left out except for the execute bit, because all files in the Nix store have 444 or 555 permission. Also, a NAR archive is _canonical_, meaning that “equal” paths always produce the same NAR archive. For instance, directory entries are always sorted so that the actual on-disk order doesn’t influence the result. This means that the cryptographic hash of a NAR dump of a path is usable as a fingerprint of the contents of the path. Indeed, the hashes of store paths stored in Nix’s database (see `nix-store --query --hash`) are SHA-256 hashes of the NAR dump of each store path. NAR archives support filenames of unlimited length and 64-bit file sizes. They can contain regular files, directories, and symbolic links, but not other types of files (such as device nodes). A Nix archive can be unpacked using [`nix-store --restore`](@docroot@/command-ref/nix-store/restore.md). {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Dump Db](https://docharvest.github.io/docs/nix/command-ref/nix-store/dump-db/) Contents nix Dump Db Nix Dump Db `nix-store --dump-db` - export Nix database # Synopsis `nix-store` `--dump-db` \[_paths…_\] # Description The operation `--dump-db` writes a dump of the Nix database to standard output. It can be loaded into an empty Nix store using `--load-db`. This is useful for making backups and when migrating to different database schemas. By default, `--dump-db` will dump the entire Nix database. When one or more store paths is passed, only the subset of the Nix database for those store paths is dumped. As with `--export`, the user is responsible for passing all the store paths for a closure. See `--export` for an example. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Export](https://docharvest.github.io/docs/nix/command-ref/nix-store/export/) Contents nix Export Nix Export `nix-store --export` - export store paths to a [Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) ## Synopsis `nix-store` `--export` _paths…_ ## Description The operation `--export` writes a serialisation of the given [store objects](@docroot@/glossary.md#gloss-store-object) to standard output in a format that can be imported into another [Nix store](@docroot@/store/index.md) with [`nix-store --import`](./import.md). > **Warning** > > This command _does not_ produce a [closure](@docroot@/glossary.md#gloss-closure) of the specified store paths. Trying to import a store object that refers to store paths not available in the target Nix store will fail. > > Use [`nix-store --query`](@docroot@/command-ref/nix-store/query.md) to obtain the closure of a store path. This command is different from [`nix-store --dump`](./dump.md), which produces a [Nix archive](@docroot@/glossary.md#gloss-nar) that _does not_ contain the set of [references](@docroot@/glossary.md#gloss-reference) of a given store path. > **Note** > > For efficient transfer of closures to remote machines over SSH, use [`nix-copy-closure`](@docroot@/command-ref/nix-copy-closure.md). {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples > **Example** > > Deploy GNU Hello to an airgapped machine via USB stick. > > Write the closure to the block device on a machine with internet connection: > > ``` > [alice@itchy]$ storePath=$(nix-build '' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link) > [alice@itchy]$ nix-store --export $(nix-store --query --requisites $storePath) | sudo dd of=/dev/usb > ``` > > Read the closure from the block device on the machine without internet connection: > > ``` > [bob@scratchy]$ hello=$(sudo dd if=/dev/usb | nix-store --import | tail -1) > [bob@scratchy]$ $hello/bin/hello > Hello, world! > ``` ## [Name](https://docharvest.github.io/docs/nix/command-ref/nix-store/gc/) Contents nix Name Nix Name `nix-store --gc` - run garbage collection # Synopsis `nix-store` `--gc` \[`--print-roots` | `--print-live` | `--print-dead`\] \[`--max-freed` _bytes_\] # Description Without additional flags, the operation `--gc` performs a garbage collection on the Nix store. That is, all paths in the Nix store not reachable via file system references from a set of “roots”, are deleted. The following suboperations may be specified: - `--print-roots` This operation prints on standard output the set of roots used by the garbage collector. - `--print-live` This operation prints on standard output the set of “live” store paths, which are all the store paths reachable from the roots. Live paths should never be deleted, since that would break consistency — it would become possible that applications are installed that reference things that are no longer present in the store. - `--print-dead` This operation prints out on standard output the set of “dead” store paths, which is just the opposite of the set of live paths: any path in the store that is not live (with respect to the roots) is dead. By default, all unreachable paths are deleted. The following options control what gets deleted and in what order: - `--max-freed` _bytes_ Keep deleting paths until at least _bytes_ bytes have been deleted, then stop. The argument _bytes_ can be followed by the multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB or TiB units. The behaviour of the collector is also influenced by the `keep-outputs` and `keep-derivations` settings in the Nix configuration file. By default, the collector prints the total number of freed bytes when it finishes (or when it is interrupted). {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples To delete all unreachable paths, just do: ``` $ nix-store --gc deleting `/nix/store/kq82idx6g0nyzsp2s14gfsc38npai7lf-cairo-1.0.4.tar.gz.drv' ... 8825586 bytes freed (8.42 MiB) ``` To delete at least 100 MiBs of unreachable paths: ``` $ nix-store --gc --max-freed $((100 * 1024 * 1024)) ``` ## [Generate Binary Cache Key](https://docharvest.github.io/docs/nix/command-ref/nix-store/generate-binary-cache-key/) Contents nix Generate Binary Cache Key Nix Generate Binary Cache Key `nix-store --generate-binary-cache-key` - generate key pair to use for a binary cache ## Synopsis `nix-store` `--generate-binary-cache-key` _key-name_ _secret-key-file_ _public-key-file_ ## Description This command generates an [Ed25519 key pair](http://ed25519.cr.yp.to/) that can be used to create a signed binary cache. It takes three mandatory parameters: 1. A key name, such as `cache.example.org-1`, that is used to look up keys on the client when it verifies signatures. It can be anything, but it’s suggested to use the host name of your cache (e.g. `cache.example.org`) with a suffix denoting the number of the key (to be incremented every time you need to revoke a key). 2. The file name where the secret key is to be stored. 3. The file name where the public key is to be stored. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Import](https://docharvest.github.io/docs/nix/command-ref/nix-store/import/) Contents nix Import Nix Import `nix-store --import` - import [Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) into the store # Synopsis `nix-store` `--import` # Description The operation `--import` reads a serialisation of a set of [store objects](@docroot@/glossary.md#gloss-store-object) produced by [`nix-store --export`](./export.md) from standard input, and adds those store objects to the specified [Nix store](@docroot@/store/index.md). Paths that already exist in the target Nix store are ignored. If a path [refers](@docroot@/glossary.md#gloss-reference) to another path that doesn’t exist in the target Nix store, the import fails. > **Note** > > For efficient transfer of closures to remote machines over SSH, use [`nix-copy-closure`](@docroot@/command-ref/nix-copy-closure.md). {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples > **Example** > > Given a closure of GNU Hello as a file: > > ``` > $ storePath="$(nix-build '' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link)" > $ nix-store --export $(nix-store --query --requisites $storePath) > hello.closure > ``` > > Import the closure into a [remote SSH store](@docroot@/store/types/ssh-store.md) using the [`--store`](@docroot@/command-ref/conf-file.md#conf-store) option: > > ``` > $ nix-store --import --store ssh://alice@itchy.example.org < hello.closure > ``` ## [Load Db](https://docharvest.github.io/docs/nix/command-ref/nix-store/load-db/) Contents nix Load Db Nix Load Db `nix-store --load-db` - import Nix database # Synopsis `nix-store` `--load-db` # Description The operation `--load-db` reads a dump of the Nix database created by `--dump-db` from standard input and loads it into the Nix database. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Opt Common](https://docharvest.github.io/docs/nix/command-ref/nix-store/opt-common/) Contents nix Opt Common Nix Opt Common The following options are allowed for all `nix-store` operations, but may not always have an effect. - [`--add-root`](#opt-add-root) _path_ Causes the result of a realisation (`--realise` and `--force-realise`) to be registered as a root of the garbage collector. _path_ will be created as a symlink to the resulting store path. In addition, a uniquely named symlink to _path_ will be created in `/nix/var/nix/gcroots/auto/`. For instance, ``` $ nix-store --add-root /home/eelco/bla/result --realise ... $ ls -l /nix/var/nix/gcroots/auto lrwxrwxrwx 1 ... 2005-03-13 21:10 dn54lcypm8f8... -> /home/eelco/bla/result $ ls -l /home/eelco/bla/result lrwxrwxrwx 1 ... 2005-03-13 21:10 /home/eelco/bla/result -> /nix/store/1r11343n6qd4...-f-spot-0.0.10 ``` Thus, when `/home/eelco/bla/result` is removed, the GC root in the `auto` directory becomes a dangling symlink and will be ignored by the collector. > **Warning** > > Note that it is not possible to move or rename GC roots, since the symlink in the `auto` directory will still point to the old location. If there are multiple results, then multiple symlinks will be created by sequentially numbering symlinks beyond the first one (e.g., `foo`, `foo-2`, `foo-3`, and so on). ## [Optimise](https://docharvest.github.io/docs/nix/command-ref/nix-store/optimise/) Contents nix Optimise Nix Optimise `nix-store --optimise` - reduce disk space usage ## Synopsis `nix-store` `--optimise` ## Description The operation `--optimise` reduces Nix store disk space usage by finding identical files in the store and hard-linking them to each other. It typically reduces the size of the store by something like 25-35%. Only regular files and symlinks are hard-linked in this manner. Files are considered identical when they have the same [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) serialisation: that is, regular files must have the same contents and permission (executable or non-executable), and symlinks must have the same contents. After completion, or when the command is interrupted, a report on the achieved savings is printed on standard error. Use `-vv` or `-vvv` to get some progress indication. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## Example ``` $ nix-store --optimise hashing files in `/nix/store/qhqx7l2f1kmwihc9bnxs7rc159hsxnf3-gcc-4.1.1' ... 541838819 bytes (516.74 MiB) freed by hard-linking 54143 files; there are 114486 files with equal contents out of 215894 files in total ``` ## [Print Env](https://docharvest.github.io/docs/nix/command-ref/nix-store/print-env/) Contents nix Print Env Nix Print Env `nix-store --print-env` - print the build environment of a derivation ## Synopsis `nix-store` `--print-env` _drvpath_ ## Description The operation `--print-env` prints out the environment of a derivation in a format that can be evaluated by a shell. The command line arguments of the builder are placed in the variable `_args`. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## Example ``` $ nix-store --print-env $(nix-instantiate '' -A firefox) … export src; src='/nix/store/plpj7qrwcz94z2psh6fchsi7s8yihc7k-firefox-12.0.source.tar.bz2' export stdenv; stdenv='/nix/store/7c8asx3yfrg5dg1gzhzyq2236zfgibnn-stdenv' export system; system='x86_64-linux' export _args; _args='-e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25c-default-builder.sh' ``` ## [Query](https://docharvest.github.io/docs/nix/command-ref/nix-store/query/) Contents nix Query Nix Query `nix-store --query` - display information about store paths # Synopsis `nix-store` {`--query` | `-q`} {`--outputs` | `--requisites` | `-R` | `--references` | `--referrers` | `--referrers-closure` | `--deriver` | `-d` | `--valid-derivers` | `--graph` | `--tree` | `--binding` _name_ | `-b` _name_ | `--hash` | `--size` | `--roots`} \[`--use-output`\] \[`-u`\] \[`--force-realise`\] \[`-f`\] _paths…_ # Description The operation `--query` displays various bits of information about the store paths . The queries are described below. At most one query can be specified. The default query is `--outputs`. The paths _paths_ may also be symlinks from outside of the Nix store, to the Nix store. In that case, the query is applied to the target of the symlink. # Common query options - `--use-output` / `-u` For each argument to the query that is a [store derivation](@docroot@/glossary.md#gloss-store-derivation), apply the query to the output path of the derivation instead. - `--force-realise` / `-f` Realise each argument to the query first (see [`nix-store --realise`](./realise.md)). # Queries - `--outputs` Prints out the [output paths](@docroot@/glossary.md#gloss-output-path) of the store derivations _paths_. These are the paths that will be produced when the derivation is built. - `--references` Prints the set of [references](@docroot@/glossary.md#gloss-reference) of the store paths _paths_, that is, their immediate dependencies. (For _all_ dependencies, use `--requisites`.) - `--requisites` / `-R` Prints out the set of [_requisites_](@docroot@/glossary.md#gloss-requisite) (better known as the [closure](@docroot@/glossary.md#gloss-closure)) of the store path _paths_. This query has one option: - `--include-outputs` Also include the existing output paths of [store derivation](@docroot@/glossary.md#gloss-store-derivation)s, and their closures. This query can be used to implement various kinds of deployment. A _source deployment_ is obtained by distributing the closure of a store derivation. A _binary deployment_ is obtained by distributing the closure of an output path. A _cache deployment_ (combined source/binary deployment, including binaries of build-time-only dependencies) is obtained by distributing the closure of a store derivation and specifying the option `--include-outputs`. - `--referrers` Prints the set of [_referrers_](@docroot@/glossary.md#gloss-referrer) of the store paths _paths_, that is, the store paths currently existing in the Nix store that refer to one of _paths_. Note that contrary to the references, the set of referrers is not constant; it can change as store paths are added or removed. - `--referrers-closure` Prints the closure of the set of store paths _paths_ under the [referrers relation](@docroot@/glossary.md#gloss-referrer); that is, all store paths that directly or indirectly refer to one of _paths_. These are all the path currently in the Nix store that are dependent on _paths_. - `--deriver` / `-d` Prints the [deriver](@docroot@/glossary.md#gloss-deriver) that was used to build the store paths _paths_. If the path has no deriver (e.g., if it is a source file), or if the deriver is not known (e.g., in the case of a binary-only deployment), the string `unknown-deriver` is printed. The returned deriver is not guaranteed to exist in the local store, for example when _paths_ were substituted from a binary cache. Use `--valid-derivers` instead to obtain valid paths only. - `--valid-derivers` Prints a set of derivation files (`.drv`) which are supposed produce said paths when realized. Might print nothing, for example for source paths or paths substituted from a binary cache. - `--graph` Prints the references graph of the store paths _paths_ in the format of the `dot` tool of AT&T's [Graphviz package](http://www.graphviz.org/). This can be used to visualise dependency graphs. To obtain a build-time dependency graph, apply this to a store derivation. To obtain a runtime dependency graph, apply it to an output path. - `--tree` Prints the references graph of the store paths _paths_ as a nested ASCII tree. References are ordered by descending closure size; this tends to flatten the tree, making it more readable. The query only recurses into a store path when it is first encountered; this prevents a blowup of the tree representation of the graph. - `--graphml` Prints the references graph of the store paths _paths_ in the [GraphML](http://graphml.graphdrawing.org/) file format. This can be used to visualise dependency graphs. To obtain a build-time dependency graph, apply this to a [store derivation](@docroot@/glossary.md#gloss-store-derivation). To obtain a runtime dependency graph, apply it to an output path. - `--binding` _name_ / `-b` _name_ Prints the value of the attribute _name_ (i.e., environment variable) of the [store derivation](@docroot@/glossary.md#gloss-store-derivation)s _paths_. It is an error for a derivation to not have the specified attribute. - `--hash` Prints the SHA-256 hash of the contents of the store paths _paths_ (that is, the hash of the output of `nix-store --dump` on the given paths). Since the hash is stored in the Nix database, this is a fast operation. - `--size` Prints the size in bytes of the contents of the store paths _paths_ — to be precise, the size of the output of `nix-store --dump` on the given paths. Note that the actual disk space required by the store paths may be higher, especially on filesystems with large cluster sizes. - `--roots` Prints the garbage collector roots that point, directly or indirectly, at the store paths _paths_. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples Print the closure (runtime dependencies) of the `svn` program in the current user environment: ``` $ nix-store --query --requisites $(which svn) /nix/store/5mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 /nix/store/9lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4 ... ``` Print the build-time dependencies of `svn`: ``` $ nix-store --query --requisites $(nix-store --query --deriver $(which svn)) /nix/store/y6qa66l9h0pw161crnlk6y16rdrcljx4-grep-2.5.1.tar.bz2.drv /nix/store/z716h753s97jhnzvfank2srqbljswpgm-gcc-wrapper.sh /nix/store/f39x0q73rjdyvzm93y9wrkfr6x39lb7f-glibc-2.3.4.drv ... lots of other paths ... ``` The difference with the previous example is that we ask the closure of the derivation (`-qd`), not the closure of the output path that contains `svn`. Show the build-time dependencies as a tree: ``` $ nix-store --query --tree $(nix-store --query --deriver $(which svn)) /nix/store/7i5082kfb6yjbqdbiwdhhza0am2xvh6c-subversion-1.1.4.drv +---/nix/store/vxnmkc8l8d2ijjha4xwhkfgx9vvc3q4c-builder.sh +---/nix/store/rn9776dy82n5qrgz7xbcl1iw4vfkcrkk-bash-3.0.drv | +---/nix/store/x9j20hz6bln1crzn55qifk0bbsm8v5ac-bash | +---/nix/store/ajnn1mcm45wjvn0rlc22gvx2cwhjnazx-builder.sh ... ``` Show all paths that depend on the same OpenSSL library as `svn`: ``` $ nix-store --query --referrers $(nix-store --query --binding openssl $(nix-store --query --deriver $(which svn))) /nix/store/23ny9l9wixx21632y2wi4p585qhva1q8-sylpheed-1.0.0 /nix/store/5mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 /nix/store/dpmvp969yhdqs7lm2r1a3gng7pyq6vy4-subversion-1.1.3 /nix/store/l51240xqsgg8a7yrbqdx1rfzyv6l26fx-lynx-2.8.5 ``` Show all paths that directly or indirectly depend on the Glibc (C library) used by `svn`: ``` $ nix-store --query --referrers-closure $(ldd $(which svn) | grep /libc.so | awk '{print $3}') /nix/store/034a6h4vpz9kds5r6kzb9lhh81mscw43-libgnomeprintui-2.8.2 /nix/store/15l3yi0d45prm7a82pcrknxdh6nzmxza-gawk-3.1.4 ... ``` Note that `ldd` is a command that prints out the dynamic libraries used by an ELF executable. Make a picture of the runtime dependency graph of the current user environment: ``` $ nix-store --query --graph ~/.nix-profile | dot -Tps > graph.ps $ gv graph.ps ``` Show every garbage collector root that points to a store path that depends on `svn`: ``` $ nix-store --query --roots $(which svn) /nix/var/nix/profiles/default-81-link /nix/var/nix/profiles/default-82-link /home/eelco/.local/state/nix/profiles/profile-97-link ``` ## [Read Log](https://docharvest.github.io/docs/nix/command-ref/nix-store/read-log/) Contents nix Read Log Nix Read Log `nix-store --read-log` - print build log # Synopsis `nix-store` {`--read-log` | `-l`} _paths…_ # Description The operation `--read-log` prints the build log of the specified store paths on standard output. The build log is whatever the builder of a derivation wrote to standard output and standard error. If a store path is not a derivation, the deriver of the store path is used. Build logs are kept in `/nix/var/log/nix/drvs`. However, there is no guarantee that a build log is available for any particular store path. For instance, if the path was downloaded as a pre-built binary through a substitute, then the log is unavailable. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Example ``` $ nix-store --read-log $(which ktorrent) building /nix/store/dhc73pvzpnzxhdgpimsd9sw39di66ph1-ktorrent-2.2.1 unpacking sources unpacking source archive /nix/store/p8n1jpqs27mgkjw07pb5269717nzf5f8-ktorrent-2.2.1.tar.gz ktorrent-2.2.1/ ktorrent-2.2.1/NEWS ... ``` ## [Realise](https://docharvest.github.io/docs/nix/command-ref/nix-store/realise/) Contents nix Realise Nix Realise `nix-store --realise` - build or fetch store objects # Synopsis `nix-store` {`--realise` | `-r`} _paths…_ \[`--dry-run`\] # Description Each of _paths_ is processed as follows: - If the path leads to a [store derivation](@docroot@/glossary.md#gloss-store-derivation): 1. If it is not [valid](@docroot@/glossary.md#gloss-validity), substitute the store derivation file itself. 2. Realise its [output paths](@docroot@/glossary.md#gloss-output-path): - Try to fetch from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters) the [store objects](@docroot@/store/store-object.md) associated with the output paths in the store derivation's [closure](@docroot@/glossary.md#gloss-closure). - With [content-addressing derivations](@docroot@/development/experimental-features.md#xp-feature-ca-derivations) (experimental): Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database](@docroot@/glossary.md#gloss-nix-database). - For any store paths that cannot be substituted, produce the required store objects: 1. Realise all outputs of the derivation's dependencies 2. Run the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable - Otherwise, and if the path is not already valid: Try to fetch the associated [store objects](@docroot@/store/store-object.md) in the path's [closure](@docroot@/glossary.md#gloss-closure) from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters). If no substitutes are available and no store derivation is given, realisation fails. The resulting paths are printed on standard output. For non-derivation arguments, the argument itself is printed. {{#include ../status-build-failure.md}} # Options - `--dry-run` Print on standard error a description of what packages would be built or downloaded, without actually performing the operation. - `--ignore-unknown` If a non-derivation path does not have a substitute, then silently ignore it. - `--check` This option allows you to check whether a derivation is deterministic. It rebuilds the specified derivation and checks whether the result is bitwise-identical with the existing outputs, printing an error if that’s not the case. The outputs of the specified derivation must already exist. When used with `-K`, if an output path is not identical to the corresponding output from the previous build, the new output path is left in `/nix/store/name.check.` {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples This operation is typically used to build [store derivation](@docroot@/glossary.md#gloss-store-derivation)s produced by [`nix-instantiate`](@docroot@/command-ref/nix-instantiate.md): ``` $ nix-store --realise $(nix-instantiate ./test.nix) /nix/store/6gwmy5jcnwdlz6aqqhksz863f1l8xc2w-aterm-2.3.1 ``` This is essentially what [`nix-build`](@docroot@/command-ref/nix-build.md) does. To test whether a previously-built derivation is deterministic: ``` $ nix-build '' --attr hello --check -K ``` Use [`nix-store --read-log`](./read-log.md) to show the stderr and stdout of a build: ``` $ nix-store --read-log $(nix-instantiate ./test.nix) ``` ## [Repair Path](https://docharvest.github.io/docs/nix/command-ref/nix-store/repair-path/) Contents nix Repair Path Nix Repair Path `nix --repair-path` - re-download path from substituter # Synopsis `nix-store` `--repair-path` _paths…_ # Description The operation `--repair-path` attempts to “repair” the specified paths by redownloading them using the available substituters. If no substitutes are available, then repair is not possible. > **Warning** > > During repair, there is a very small time window during which the old path (if it exists) is moved out of the way and replaced with the new path. If repair is interrupted in between, then the system may be left in a broken state (e.g., if the path contains a critical system component like the GNU C Library). # Example ``` $ nix-store --verify-path /nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 path `/nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13' was modified! expected hash `2db57715ae90b7e31ff1f2ecb8c12ec1cc43da920efcbe3b22763f36a1861588', got `481c5aa5483ebc97c20457bb8bca24deea56550d3985cda0027f67fe54b808e4' $ nix-store --repair-path /nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 fetching path `/nix/store/d7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13'... … ``` ## [Restore](https://docharvest.github.io/docs/nix/command-ref/nix-store/restore/) Contents nix Restore Nix Restore `nix-store --restore` - extract a Nix archive ## Synopsis `nix-store` `--restore` _path_ ## Description The operation `--restore` unpacks a [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) to _path_, which must not already exist. The archive is read from standard input. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Serve](https://docharvest.github.io/docs/nix/command-ref/nix-store/serve/) Contents nix Serve Nix Serve `nix-store --serve` - serve local Nix store over SSH # Synopsis `nix-store` `--serve` \[`--write`\] # Description The operation `--serve` provides access to the Nix store over stdin and stdout, and is intended to be used as a means of providing Nix store access to a restricted ssh user. The following flags are available: - `--write` Allow the connected client to request the realization of derivations. In effect, this can be used to make the host act as a remote builder. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} # Examples To turn a host into a build server, the `authorized_keys` file can be used to provide build access to a given SSH public key: ``` $ cat <>/root/.ssh/authorized_keys command="nice -n20 nix-store --serve --write" ssh-rsa AAAAB3NzaC1yc2EAAAA... EOF ``` ## [Verify](https://docharvest.github.io/docs/nix/command-ref/nix-store/verify/) Contents nix Verify Nix Verify `nix-store --verify` - check Nix database for consistency # Synopsis `nix-store` `--verify` \[`--check-contents`\] \[`--repair`\] # Description The operation `--verify` verifies the internal consistency of the Nix database, and the consistency between the Nix database and the Nix store. Any inconsistencies encountered are automatically repaired. Inconsistencies are generally the result of the Nix store or database being modified by non-Nix tools, or of bugs in Nix itself. This operation has the following options: - `--check-contents` Checks that the contents of every valid store path has not been altered by computing a SHA-256 hash of the contents and comparing it with the hash stored in the Nix database at build time. Paths that have been modified are printed out. For large stores, `--check-contents` is obviously quite slow. - `--repair` If any valid path is missing from the store, or (if `--check-contents` is given) the contents of a valid path has been modified, then try to repair the path by redownloading it. See `nix-store --repair-path` for details. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## [Verify Path](https://docharvest.github.io/docs/nix/command-ref/nix-store/verify-path/) Contents nix Verify Path Nix Verify Path `nix-store --verify-path` - check path contents against Nix database ## Synopsis `nix-store` `--verify-path` _paths…_ ## Description The operation `--verify-path` compares the contents of the given store paths to their cryptographic hashes stored in Nix’s database. For every changed path, it prints a warning message. The exit status is 0 if no path has changed, and 1 otherwise. {{#include ./opt-common.md}} {{#include ../opt-common.md}} {{#include ../env-common.md}} ## Example To verify the integrity of the `svn` command and all its dependencies: ``` $ nix-store --verify-path $(nix-store --query --requisites $(which svn)) ``` ## [Opt Common](https://docharvest.github.io/docs/nix/command-ref/opt-common/) Contents nix Opt Common Nix Opt Common # Common Options Most Nix commands accept the following command-line options: - [`--help`](#opt-help) Prints out a summary of the command syntax and exits. - [`--version`](#opt-version) Prints out the Nix version number on standard output and exits. - [`--verbose`](#opt-verbose) / `-v` Increases the level of verbosity of diagnostic messages printed on standard error. For each Nix operation, the information printed on standard output is well-defined; any diagnostic information is printed on standard error, never on standard output. This option may be specified repeatedly. Currently, the following verbosity levels exist: - `0` “Errors only” Only print messages explaining why the Nix invocation failed. - `1` “Informational” Print _useful_ messages about what Nix is doing. This is the default. - `2` “Talkative” Print more informational messages. - `3` “Chatty” Print even more informational messages. - `4` “Debug” Print debug information. - `5` “Vomit” Print vast amounts of debug information. - [`--quiet`](#opt-quiet) Decreases the level of verbosity of diagnostic messages printed on standard error. This is the inverse option to `-v` / `--verbose`. This option may be specified repeatedly. See the previous verbosity levels list. - [`--log-format`](#opt-log-format) _format_ This option can be used to change the output of the log format, with _format_ being one of: - `raw` This is the raw format, as outputted by nix-build. - `internal-json` Outputs the logs in a structured manner. > **Warning** > > While the schema itself is relatively stable, the format of the error-messages (namely of the `msg`\-field) can change between releases. - `bar` Only display a progress bar during the builds. - `bar-with-logs` Display the raw logs, with the progress bar at the bottom. - [`--no-build-output`](#opt-no-build-output) / `-Q` By default, output written by builders to standard output and standard error is echoed to the Nix command's standard error. This option suppresses this behaviour. Note that the builder's standard output and error are always written to a log file in `prefix/nix/var/log/nix`. - [`--max-jobs`](#opt-max-jobs) / `-j` _number_ Sets the maximum number of build jobs that Nix will perform in parallel to the specified number. Specify `auto` to use the number of CPUs in the system. The default is specified by the `max-jobs` configuration setting, which itself defaults to `1`. A higher value is useful on SMP systems or to exploit I/O latency. Setting it to `0` disallows building on the local machine, which is useful when you want builds to happen only on remote builders. - [`--cores`](#opt-cores) Sets the value of the `NIX_BUILD_CORES` environment variable in the invocation of builders. Builders can use this variable at their discretion to control the maximum amount of parallelism. For instance, in Nixpkgs, if the derivation attribute `enableParallelBuilding` is set to `true`, the builder passes the `-jN` flag to GNU Make. It defaults to the value of the `cores` configuration setting, if set, or `1` otherwise. The value `0` means that the builder should use all available CPU cores in the system. - [`--max-silent-time`](#opt-max-silent-time) Sets the maximum number of seconds that a builder can go without producing any data on standard output or standard error. The default is specified by the `max-silent-time` configuration setting. `0` means no time-out. - [`--timeout`](#opt-timeout) Sets the maximum number of seconds that a builder can run. The default is specified by the `timeout` configuration setting. `0` means no timeout. - [`--keep-going`](#opt-keep-going) / `-k` Keep going in case of failed builds, to the greatest extent possible. That is, if building an input of some derivation fails, Nix will still build the other inputs, but not the derivation itself. Without this option, Nix stops if any build fails (except for builds of substitutes), possibly killing builds in progress (in case of parallel or distributed builds). - [`--keep-failed`](#opt-keep-failed) / `-K` Specifies that in case of a build failure, the temporary directory (usually in `/tmp`) in which the build takes place should not be deleted. The path of the build directory is printed as an informational message. - [`--fallback`](#opt-fallback) Whenever Nix attempts to build a derivation for which substitutes are known for each output path, but realising the output paths through the substitutes fails, fall back on building the derivation. The most common scenario in which this is useful is when we have registered substitutes in order to perform binary distribution from, say, a network repository. If the repository is down, the realisation of the derivation will fail. When this option is specified, Nix will build the derivation instead. Thus, installation from binaries falls back on installation from source. This option is not the default since it is generally not desirable for a transient failure in obtaining the substitutes to lead to a full build from source (with the related consumption of resources). - [`--readonly-mode`](#opt-readonly-mode) When this option is used, no attempt is made to open the Nix database. Most Nix operations do need database access, so those operations will fail. - [`--arg`](#opt-arg) _name_ _value_ This option is accepted by `nix-env`, `nix-instantiate`, `nix-shell` and `nix-build`. When evaluating Nix expressions, the expression evaluator will automatically try to call functions that it encounters. It can automatically call functions for which every argument has a [default value](@docroot@/language/syntax.md#functions) (e.g., `{ argName ? defaultValue }: ...`). With `--arg`, you can also call functions that have arguments without a default value (or override a default value). That is, if the evaluator encounters a function with an argument named _name_, it will call it with value _value_. For instance, the top-level `default.nix` in Nixpkgs is actually a function: ``` { # The system (e.g., `i686-linux') for which to build the packages. system ? builtins.currentSystem ... }: ... ``` So if you call this Nix expression (e.g., when you do `nix-env --install --attr pkgname`), the function will be called automatically using the value [`builtins.currentSystem`](@docroot@/language/builtins.md) for the `system` argument. You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`. (Note that since the argument is a Nix string literal, you have to escape the quotes.) - [`--arg-from-file`](#opt-arg-from-file) _name_ _path_ Pass the contents of file _path_ as the argument _name_ to Nix functions. - [`--arg-from-stdin`](#opt-arg-from-stdin) _name_ Pass the contents of stdin as the argument _name_ to Nix functions. - [`--argstr`](#opt-argstr) _name_ _value_ This option is like `--arg`, only the value is not a Nix expression but a string. So instead of `--arg system \"i686-linux\"` (the outer quotes are to keep the shell happy) you can say `--argstr system i686-linux`. - [`--attr`](#opt-attr) / `-A` _attrPath_ Select an attribute from the top-level Nix expression being evaluated. (`nix-env`, `nix-instantiate`, `nix-build` and `nix-shell` only.) The _attribute path_ _attrPath_ is a sequence of attribute names separated by dots. For instance, given a top-level Nix expression _e_, the attribute path `xorg.xorgserver` would cause the expression `e.xorg.xorgserver` to be used. See [`nix-env --install`](@docroot@/command-ref/nix-env/install.md) for some concrete examples. In addition to attribute names, you can also specify array indices. For instance, the attribute path `foo.3.bar` selects the `bar` attribute of the fourth element of the array in the `foo` attribute of the top-level expression. - [`--eval-store`](#opt-eval-store) _store-url_ The [URL to the Nix store](@docroot@/store/types/index.md#store-url-format) to use for evaluation, i.e. where to store derivations (`.drv` files) and inputs referenced by them. - [`--expr`](#opt-expr) / `-E` Interpret the command line arguments as a list of Nix expressions to be parsed and evaluated, rather than as a list of file names of Nix expressions. (`nix-instantiate`, `nix-build` and `nix-shell` only.) For `nix-shell`, this option is commonly used to give you a shell in which you can build the packages returned by the expression. If you want to get a shell which contain the _built_ packages ready for use, give your expression to the `nix-shell --packages` convenience flag instead. - [`-I` / `--include`](#opt-I) _path_ Add an entry to the list of search paths used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md). This option may be given multiple times. Paths added through `-I` take precedence over the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path) and the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH). - [`--impure`](#opt-impure) Allow access to mutable paths and repositories. - [`--option`](#opt-option) _name_ _value_ Set the Nix configuration option _name_ to _value_. This overrides settings in the Nix configuration file (see nix.conf5). - [`--repair`](#opt-repair) Fix corrupted or missing store paths by redownloading or rebuilding them. Note that this is slow because it requires computing a cryptographic hash of the contents of every path in the closure of the build. Also note the warning under `nix-store --repair-path`. > **Note** > > See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags. ## [Special exit codes for build failure](https://docharvest.github.io/docs/nix/command-ref/status-build-failure/) Contents nix Special exit codes for build failure Nix Special exit codes for build failure 1xx status codes are used when requested builds failed. The following codes are in use: - `100` Generic build failure The builder process returned with a non-zero exit code. - `101` Build timeout The build was aborted because it did not complete within the specified `timeout`. - `102` Hash mismatch The build output was rejected because it does not match the [`outputHash` attribute of the derivation](@docroot@/language/advanced-attributes.md). - `104` Not deterministic The build succeeded in check mode but the resulting output is not binary reproducible. With the `--keep-going` flag it's possible for multiple failures to occur. In this case the 1xx status codes are or combined using [bitwise OR](https://en.wikipedia.org/wiki/Bitwise_operation#OR). ``` 0b1100100 ^^^^ |||`- timeout ||`-- output hash mismatch |`--- build failure `---- not deterministic ``` ## [Utilities](https://docharvest.github.io/docs/nix/command-ref/utilities/) Contents nix Utilities Nix Utilities This section lists utilities that you can use when you work with Nix. ## [Development](https://docharvest.github.io/docs/nix/development/) Contents nix Development Nix Development Nix is developed on GitHub. Check the [contributing guide](https://github.com/NixOS/nix/blob/master/CONTRIBUTING.md) if you want to get involved. This chapter is a collection of guides for making changes to the code and documentation. If you're not sure where to start, try to [compile Nix from source](./building.md) and consider [making improvements to documentation](./documentation.md). ## [Running Benchmarks](https://docharvest.github.io/docs/nix/development/benchmarking/) Contents nix Running Benchmarks Nix Running Benchmarks This guide explains how to build and run performance benchmarks in the Nix codebase. ## Overview Nix uses the [Google Benchmark](https://github.com/google/benchmark) framework for performance testing. Benchmarks help measure and track the performance of critical operations like derivation parsing. ## Building Benchmarks Benchmarks are disabled by default and must be explicitly enabled during the build configuration. For accurate results, use a debug-optimized release build. ### Development Environment Setup First, enter the development shell which includes the necessary dependencies: ``` nix develop .#native-ccacheStdenv ``` ### Configure Build with Benchmarks From the project root, configure the build with benchmarks enabled and optimization: ``` cd build meson configure -Dbenchmarks=true -Dbuildtype=debugoptimized ``` The `debugoptimized` build type provides: - Compiler optimizations for realistic performance measurements - Debug symbols for profiling and analysis - Balance between performance and debuggability ### Build the Benchmarks Build the project including benchmarks: ``` ninja ``` This will create benchmark executables in the build directory. Currently available: - `build/src/libstore-tests/nix-store-benchmarks` - Store-related performance benchmarks Additional benchmark executables will be created as more benchmarks are added to the codebase. ## Running Benchmarks ### Basic Usage Run benchmark executables directly. For example, to run store benchmarks: ``` ./build/src/libstore-tests/nix-store-benchmarks ``` As more benchmark executables are added, run them similarly from their respective build directories. ### Filtering Benchmarks Run specific benchmarks using regex patterns: ``` # Run only derivation parser benchmarks ./build/src/libstore-tests/nix-store-benchmarks --benchmark_filter="derivation.*" # Run only benchmarks for hello.drv ./build/src/libstore-tests/nix-store-benchmarks --benchmark_filter=".*hello.*" ``` ### Output Formats Generate benchmark results in different formats: ``` # JSON output ./build/src/libstore-tests/nix-store-benchmarks --benchmark_format=json > results.json # CSV output ./build/src/libstore-tests/nix-store-benchmarks --benchmark_format=csv > results.csv ``` ### Advanced Options ``` # Run benchmarks multiple times for better statistics ./build/src/libstore-tests/nix-store-benchmarks --benchmark_repetitions=10 # Set minimum benchmark time (useful for micro-benchmarks) ./build/src/libstore-tests/nix-store-benchmarks --benchmark_min_time=2 # Compare against baseline ./build/src/libstore-tests/nix-store-benchmarks --benchmark_baseline=baseline.json # Display time in custom units ./build/src/libstore-tests/nix-store-benchmarks --benchmark_time_unit=ms ``` ## Writing New Benchmarks To add new benchmarks: 1. Create a new `.cc` file in the appropriate `*-tests` directory 2. Include the benchmark header: ``` #include ``` 3. Write benchmark functions: ``` static void BM_YourBenchmark(benchmark::State & state) { // Setup code here for (auto _ : state) { // Code to benchmark } } BENCHMARK(BM_YourBenchmark); ``` 4. Add the file to the corresponding `meson.build`: ``` benchmarks_sources = files( 'your-benchmark.cc', # existing benchmarks... ) ``` ## Profiling with Benchmarks For deeper performance analysis, combine benchmarks with profiling tools: ``` # Using Linux perf perf record ./build/src/libstore-tests/nix-store-benchmarks perf report ``` ### Using Valgrind Callgrind Valgrind's callgrind tool provides detailed profiling information that can be visualized with kcachegrind: ``` # Profile with callgrind valgrind --tool=callgrind ./build/src/libstore-tests/nix-store-benchmarks # Visualize the results with kcachegrind kcachegrind callgrind.out.* ``` This provides: - Function call graphs - Instruction-level profiling - Source code annotation - Interactive visualization of performance bottlenecks ## Continuous Performance Testing ``` # Save baseline results ./build/src/libstore-tests/nix-store-benchmarks --benchmark_format=json > baseline.json # Compare against baseline in CI ./build/src/libstore-tests/nix-store-benchmarks --benchmark_baseline=baseline.json ``` ## Troubleshooting ### Benchmarks not building Ensure benchmarks are enabled: ``` meson configure build | grep benchmarks # Should show: benchmarks true ``` ### Inconsistent results - Ensure your system is not under heavy load - Disable CPU frequency scaling for consistent results - Run benchmarks multiple times with `--benchmark_repetitions` ## See Also - [Google Benchmark documentation](https://github.com/google/benchmark/blob/main/docs/user_guide.md) ## [Building Nix](https://docharvest.github.io/docs/nix/development/building/) Contents nix Building Nix Nix Building Nix This section provides some notes on how to start hacking on Nix. To get the latest version of Nix from GitHub: > **Note** > > When checking out the repo on Windows, make sure you have the git setting `core.symlinks` enabled, before cloning, as there are symlinks in the repo. ``` $ git clone https://github.com/NixOS/nix.git $ cd nix ``` > **Note** > > The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the [installation instructions](../installation/index.md). To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found: ``` $ nix-shell ``` To get a shell with one of the other [supported compilation environments](#compilation-environments): ``` $ nix-shell --attr devShells.x86_64-linux.native-clangStdenv ``` > **Note** > > You can use `native-ccacheStdenv` to drastically improve rebuild time. By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`. To build Nix itself in this shell: ``` [nix-shell]$ out="$(pwd)/outputs/out" dev=$out debug=$out mesonFlags+=" --prefix=${out}" [nix-shell]$ dontAddPrefix=1 configurePhase [nix-shell]$ buildPhase ``` To test it: ``` [nix-shell]$ checkPhase ``` To install it in `$(pwd)/outputs`: ``` [nix-shell]$ installPhase [nix-shell]$ ./outputs/out/bin/nix --version nix (Nix) 2.12 ``` To build a release version of Nix for the current operating system and CPU architecture: ``` $ nix-build ``` You can also build Nix for one of the [supported platforms](#platforms). ## Building Nix with flakes This section assumes you are using Nix with the [`flakes`](@docroot@/development/experimental-features.md#xp-feature-flakes) and [`nix-command`](@docroot@/development/experimental-features.md#xp-feature-nix-command) experimental features enabled. To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found: ``` $ nix develop ``` This shell also adds `./outputs/bin/nix` to your `$PATH` so you can run `nix` immediately after building it. To get a shell with one of the other [supported compilation environments](#compilation-environments): ``` $ nix develop .#native-clangStdenv ``` > **Note** > > Use `ccacheStdenv` to drastically improve rebuild time. By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`. To build Nix itself in this shell: ``` [nix-shell]$ configurePhase [nix-shell]$ buildPhase ``` To test it: ``` [nix-shell]$ checkPhase ``` To install it in `$(pwd)/outputs`: ``` [nix-shell]$ installPhase [nix-shell]$ nix --version nix (Nix) 2.12 ``` For more information on running and filtering tests, see [`testing.md`](./testing.md). To build a release version of Nix for the current operating system and CPU architecture: ``` $ nix build ``` You can also build Nix for one of the [supported platforms](#platforms). ## Platforms Nix can be built for various platforms, as specified in [`flake.nix`](https://github.com/nixos/nix/blob/master/flake.nix): - `x86_64-linux` - `x86_64-darwin` - `i686-linux` - `aarch64-linux` - `aarch64-darwin` - `armv6l-linux` - `armv7l-linux` - `riscv64-linux` In order to build Nix for a different platform than the one you're currently on, you need a way for your current Nix installation to build code for that platform. Common solutions include \[remote build machines\] and [binary format emulation](https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems) (only supported on NixOS). Given such a setup, executing the build only requires selecting the respective attribute. For example, to compile for `aarch64-linux`: ``` $ nix-build --attr packages.aarch64-linux.default ``` or for Nix with the [`flakes`](@docroot@/development/experimental-features.md#xp-feature-flakes) and [`nix-command`](@docroot@/development/experimental-features.md#xp-feature-nix-command) experimental features enabled: ``` $ nix build .#packages.aarch64-linux.default ``` Cross-compiled builds are available for: - `armv6l-linux` - `armv7l-linux` - `riscv64-linux` Add more [system types](#system-type) to `crossSystems` in `flake.nix` to bootstrap Nix on unsupported platforms. ### Building for multiple platforms at once It is useful to perform multiple cross and native builds on the same source tree, for example to ensure that better support for one platform doesn't break the build for another. Meson thankfully makes this very easy by confining all build products to the build directory --- one simple shares the source directory between multiple build directories, each of which contains the build for Nix to a different platform. Here's how to do that: 1. Instruct Nixpkgs's infra where we want Meson to put its build directory ``` mesonBuildDir=build-my-variant-name ``` 2. Configure as usual ``` configurePhase ``` 3. Build as usual ``` buildPhase ``` ## System type Nix uses a string with the following format to identify the _system type_ or _platform_ it runs on: ``` -[-] ``` It is set when Nix is compiled for the given system, and based on the output of Meson's [`host_machine` information](https://mesonbuild.com/Reference-manual_builtin_host_machine.html). For historic reasons and backward-compatibility, some CPU and OS identifiers are translated as follows: `host_machine.cpu_family()` `host_machine.endian()` Nix `x86` `i686` `arm` `host_machine.cpu()` `ppc` `little` `powerpcle` `ppc64` `little` `powerpc64le` `ppc` `big` `powerpc` `ppc64` `big` `powerpc64` `mips` `little` `mipsel` `mips64` `little` `mips64el` `mips` `big` `mips` `mips64` `big` `mips64` When cross-compiling Nix with Meson for local development, you need to specify a [cross-file](https://mesonbuild.com/Cross-compilation.html) using the `--cross-file` option. Cross-files define the target architecture and toolchain. When cross-compiling Nix with Nix, Nixpkgs takes care of this for you. In the nix flake we also have some cross-compilation targets available: ``` nix build .#nix-everything-riscv64-unknown-linux-gnu nix build .#nix-everything-armv7l-unknown-linux-gnueabihf nix build .#nix-everything-armv7l-unknown-linux-gnueabihf nix build .#nix-everything-x86_64-unknown-freebsd nix build .#nix-everything-x86_64-w64-mingw32 ``` ## Compilation environments Nix can be compiled using multiple environments: - `stdenv`: default; - `gccStdenv`: force the use of `gcc` compiler; - `clangStdenv`: force the use of `clang` compiler; - `ccacheStdenv`: enable \[ccache\], a compiler cache to speed up compilation. To build with one of those environments, you can use ``` $ nix build .#nix-cli-ccacheStdenv ``` for flake-enabled Nix, or ``` $ nix-build --attr nix-cli-ccacheStdenv ``` for classic Nix. You can use any of the other supported environments in place of `nix-cli-ccacheStdenv`. ## Editor integration The `clangd` LSP server is installed by default on the `clang`\-based `devShell`s. See [supported compilation environments](#compilation-environments) and instructions how to set up a shell [with flakes](#building-nix-with-flakes) or in [classic Nix](#building-nix). To use the LSP with your editor, you will want a `compile_commands.json` file telling `clangd` how we are compiling the code. Meson's configure always produces this inside the build directory. Configure your editor to use the `clangd` from the `.#native-clangStdenv` shell. You can do that either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration). > **Note** > > For some editors (e.g. Visual Studio Code), you may need to install a [special extension](https://open-vsx.org/extension/llvm-vs-code-extensions/vscode-clangd) for the editor to interact with `clangd`. Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim). Editor-specific setup is typically opinionated, so we will not cover it here in more detail. ## Formatting and pre-commit hooks You may run the formatters as a one-off using: ``` ./maintainers/format.sh ``` ### Pre-commit hooks If you'd like to run the formatters before every commit, install the hooks: ``` pre-commit-hooks-install ``` This installs [pre-commit](https://pre-commit.com) using [cachix/git-hooks.nix](https://github.com/cachix/git-hooks.nix). When making a commit, pay attention to the console output. If it fails, run `git add --patch` to approve the suggestions _and commit again_. To refresh pre-commit hook's config file, do the following: 1. Exit the development shell and start it again by running `nix develop`. 2. If you also use the pre-commit hook, also run `pre-commit-hooks-install` again. ### VSCode Insert the following json into your `.vscode/settings.json` file to configure `nixfmt`. This will be picked up by the _Format Document_ command, `"editor.formatOnSave"`, etc. ``` { "nix.formatterPath": "nixfmt", "nix.serverSettings": { "nixd": { "formatting": { "command": [ "nixfmt" ], }, }, "nil": { "formatting": { "command": [ "nixfmt" ], }, }, }, } ``` ## [CLI guideline](https://docharvest.github.io/docs/nix/development/cli-guideline/) Contents nix CLI guideline Nix CLI guideline ## Goals Purpose of this document is to provide a clear direction to **help design delightful command line** experience. This document contains guidelines to follow to ensure a consistent and approachable user experience. ## Overview `nix` command provides a single entry to a number of sub-commands that help **developers and system administrators** in the life-cycle of a software project. We particularly need to pay special attention to help and assist new users of Nix. # Naming the `COMMANDS` Words matter. Naming is an important part of the usability. Users will be interacting with Nix on a regular basis so we should **name things for ease of understanding**. We recommend following the [Principle of Least Astonishment](https://en.wikipedia.org/wiki/Principle_of_least_astonishment). This means that you should **never use acronyms or abbreviations** unless they are commonly used in other tools (e.g. `nix init`). And if the command name is too long (> 10-12 characters) then shortening it makes sense (e.g. “prioritization” → “priority”). Commands should **follow a noun-verb dialogue**. Although noun-verb formatting seems backwards from a speaking perspective (i.e. `nix store copy` vs. `nix copy store`) it allows us to organize commands the same way users think about completing an action (the group first, then the command). ## Naming rules Rules are there to guide you by limiting your options. But not everything can fit the rules all the time. In those cases document the exceptions in [Appendix 1: Commands naming exceptions](#appendix-1-commands-naming-exceptions) and provide reason. The rules want to force a Nix developer to look, not just at the command at hand, but also the command in a full context alongside other `nix` commands. ``` $ nix [] [] [] ``` - `GROUP`, `COMMAND`, `ARGUMENTS` and `OPTIONS` should be lowercase and in a singular form. - `GROUP` should be a **NOUN**. - `COMMAND` should be a **VERB**. - `ARGUMENTS` and `OPTIONS` are discussed in [_Input_ section](#input). ## Classification Some commands are more important, some less. While we want all of our commands to be perfect we can only spend limited amount of time testing and improving them. This classification tries to separate commands in 3 categories in terms of their importance in regards to the new users. Users who are likely to be impacted the most by bad user experience. - **Main commands** Commands used for our main use cases and most likely used by new users. We expect attention to details, such as: - Proper use of [colors](#colors), [emojis](#special-unicode-characters) and [aligning of text](#text-alignment). - [Autocomplete](#shell-completion) of options. - Show [next possible steps](#next-steps). - Showing some [“tips”](#educate-the-user) when running logs running tasks (eg. building / downloading) in order to teach users interesting bits of Nix ecosystem. - [Help pages](#help-is-essential) to be as good as we can write them pointing to external documentation and tutorials for more. Examples of such commands: `nix init`, `nix develop`, `nix build`, `nix run`, ... - **Infrequently used commands** From infrequently used commands we expect less attention to details, but still some: - Proper use of [colors](#colors), [emojis](#special-unicode-characters) and [aligning of text](#text-alignment). - [Autocomplete](#shell-completion) of options. Examples of such commands: `nix edit`, `nix eval`, ... - **Utility and scripting commands** Commands that expose certain internal functionality of `nix`, mostly used by other scripts. - [Autocomplete](#shell-completion) of options. Examples of such commands: `nix store copy`, `nix hash base16`, `nix store ping`, ... # Help is essential Help should be built into your command line so that new users can gradually discover new features when they need them. ## Looking for help Since there is no standard way how user will look for help we rely on ways help is provided by commonly used tools. As a guide for this we took `git` and whenever in doubt look at it as a preferred direction. The rules are: - Help is shown by using `--help` or `help` command (eg `nix` `--``help` or `nix help`). - For non-COMMANDs (eg. `nix` `--``help` and `nix store` `--``help`) we **show a summary** of most common use cases. Summary is presented on the STDOUT without any use of PAGER. - For COMMANDs (eg. `nix init` `--``help` or `nix help init`) we display the man page of that command. By default the PAGER is used (as in `git`). - At the end of either summary or man page there should be an URL pointing to an online version of more detailed documentation. - The structure of summaries and man pages should be the same as in `git`. ## Anticipate where help is needed Even better then requiring the user to search for help is to anticipate and predict when user might need it. Either because the lack of discoverability, typo in the input or simply taking the opportunity to teach the user of interesting - but less visible - details. ### Shell completion This type of help is most common and almost expected by users. We need to **provide the best shell completion** for `bash`, `zsh` and `fish`. Completion needs to be **context aware**, this mean when a user types: ``` $ nix build n ``` we need to display a list of flakes starting with `n`. ### Wrong input As we all know we humans make mistakes, all the time. When a typo - intentional or unintentional - is made, we should prompt for closest possible options or point to the documentation which would educate user to not make the same errors. Here are few examples: In first example we prompt the user for typing wrong command name: ``` $ nix int ------------------------------------------------------------------------ Error! Command `int` not found. ------------------------------------------------------------------------ Did you mean: |> nix init |> nix input ``` Sometimes users will make mistake either because of a typo or simply because of lack of discoverability. Our handling of this cases needs to be context sensitive. ``` $ nix init --template=template#python ------------------------------------------------------------------------ Error! Template `template#python` not found. ------------------------------------------------------------------------ Initializing Nix project at `/path/to/here`. Select a template for you new project: |> template#python template#python-pip template#python-poetry ``` ### Next steps It can be invaluable to newcomers to show what a possible next steps and what is the usual development workflow with Nix. For example: ``` $ nix init --template=template#python Initializing project `template#python` in `/home/USER/dev/new-project` Next steps |> nix develop -- to enter development environment |> nix build -- to build your project ``` ### Educate the user We should take any opportunity to **educate users**, but at the same time we must **be very very careful to not annoy users**. There is a thin line between being helpful and being annoying. An example of educating users might be to provide _Tips_ in places where they are waiting. ``` $ nix build Started building my-project 1.2.3 Downloaded python3.8-poetry 1.2.3 in 5.3 seconds Downloaded python3.8-requests 1.2.3 in 5.3 seconds ------------------------------------------------------------------------ Press `v` to increase logs verbosity |> `?` to see other options ------------------------------------------------------------------------ Learn something new with every build... |> See last logs of a build with `nix log --last` command. ------------------------------------------------------------------------ Evaluated my-project 1.2.3 in 14.43 seconds Downloading [12 / 200] |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left Building [2 / 20] |> glibc 1.2.3 -> buildPhase: ------------------------------------------------------------------------ ``` Now **Learn** part of the output is where you educate users. You should only show it when you know that a build will take some time and not annoy users of the builds that take only few seconds. Every feature like this should go through an intensive review and testing to collect as much feedback as possible and to fine tune every little detail. If done right this can be an awesome features beginners and advance users will love, but if not done perfectly it will annoy users and leave bad impression. # Input Input to a command is provided via `ARGUMENTS` and `OPTIONS`. `ARGUMENTS` represent a required input for a function. When choosing to use `ARGUMENTS` over `OPTIONS` please be aware of the downsides that come with it: - User will need to remember the order of `ARGUMENTS`. This is not a problem if there is only one `ARGUMENT`. - With `OPTIONS` it is possible to provide much better auto completion. - With `OPTIONS` it is possible to provide much better error message. - Using `OPTIONS` it will mean there is a little bit more typing. We don’t discourage the use of `ARGUMENTS`, but simply want to make every developer consider the downsides and choose wisely. ## Naming the `OPTIONS` The only naming convention - apart from the ones mentioned in Naming the `COMMANDS` section is how flags are named. Flags are a type of `OPTION` that represent an option that can be turned ON of OFF. We can say **flags are boolean type of** `**OPTION**`. Here are few examples of flag `OPTIONS`: - `--colors` vs. `--no-colors` (showing colors in the output) - `--emojis` vs. `--no-emojis` (showing emojis in the output) ## Prompt when input not provided For _main commands_ (as [per classification](#classification)) we want command to improve the discoverability of possible input. A new user will most likely not know which `ARGUMENTS` and `OPTIONS` are required or which values are possible for those options. In case the user does not provide the input or they provide wrong input, rather than show the error, prompt a user with an option to find and select correct input (see examples). Prompting is of course not required when TTY is not attached to STDIN. This would mean that scripts won't need to handle prompt, but rather handle errors. A place to use prompt and provide user with interactive select ``` $ nix init Initializing Nix project at `/path/to/here`. Select a template for you new project: |> py template#python-pip template#python-poetry [ Showing 2 templates from 1345 templates ] ``` Another great place to add prompts are **confirmation dialogues for dangerous actions**. For example when adding new substitutor via `OPTIONS` or via `flake.nix` we should prompt - for the first time - and let user review what is going to happen. ``` $ nix build --option substitutors https://cache.example.org ------------------------------------------------------------------------ Warning! A security related question needs to be answered. ------------------------------------------------------------------------ The following substitutors will be used to in `my-project`: - https://cache.example.org Do you allow `my-project` to use above mentioned substitutors? [y/N] |> y ``` # Output Terminal output can be quite limiting in many ways. Which should force us to think about the experience even more. As with every design the output is a compromise between being terse and being verbose, between showing help to beginners and annoying advance users. For this it is important that we know what are the priorities. Nix command line should be first and foremost written with beginners in mind. But users won't stay beginners for long and what was once useful might quickly become annoying. There is no golden rule that we can give in this guideline that would make it easier how to draw a line and find best compromise. What we would encourage is to **build prototypes**, do some **user testing** and collect **feedback**. Then repeat the cycle few times. First design the _happy path_ and only after your iron it out, continue to work on **edge cases** (handling and displaying errors, changes of the output by certain `OPTIONS`, etc…) ## Follow best practices Needless to say we Nix must be a good citizen and follow best practices in command line. In short: **STDOUT is for output, STDERR is for (human) messaging.** STDOUT and STDERR provide a way for you to output messages to the user while also allowing them to redirect content to a file. For example: ``` $ nix build > build.txt ------------------------------------------------------------------------ Error! Attribute `bin` missing at (1:94) from string. ------------------------------------------------------------------------ 1| with import { }; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (surge.bin) ]; } "" ``` Because this warning is on STDERR, it doesn’t end up in the file. But not everything on STDERR is an error though. For example, you can run `nix build` and collect logs in a file while still seeing the progress. ``` $ nix build > build.txt Evaluated 1234 files in 1.2 seconds Downloaded python3.8-poetry 1.2.3 in 5.3 seconds Downloaded python3.8-requests 1.2.3 in 5.3 seconds ------------------------------------------------------------------------ Press `v` to increase logs verbosity |> `?` to see other options ------------------------------------------------------------------------ Learn something new with every build... |> See last logs of a build with `nix log --last` command. ------------------------------------------------------------------------ Evaluated my-project 1.2.3 in 14.43 seconds Downloading [12 / 200] |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left Building [2 / 20] |> glibc 1.2.3 -> buildPhase: ------------------------------------------------------------------------ ``` ## Errors (WIP) **TODO**: Once we have implementation for the _happy path_ then we will think how to present errors. ## Not only for humans Terse, machine-readable output formats can also be useful but shouldn’t get in the way of making beautiful CLI output. When needed, commands should offer a `--json` flag to allow users to easily parse and script the CLI. When TTY is not detected on STDOUT we should remove all design elements (no colors, no emojis and using ASCII instead of Unicode symbols). The same should happen when TTY is not detected on STDERR. We should not display progress / status section, but only print warnings and errors. ## Dialog with the user CLIs don't always make it clear when an action has taken place. For every action a user performs, your CLI should provide an equal and appropriate reaction, clearly highlighting the what just happened. For example: ``` $ nix build Downloaded python3.8-poetry 1.2.3 in 5.3 seconds Downloaded python3.8-requests 1.2.3 in 5.3 seconds ... Success! You have successfully built my-project. $ ``` Above command clearly states that command successfully completed. And in case of `nix build`, which is a command that might take some time to complete, it is equally important to also show that a command started. ## Text alignment Text alignment is the number one design element that will present all of the Nix commands as a family and not as separate tools glued together. The format we should follow is: ``` $ nix COMMAND VERB_1 NOUN and other words VERB__1 NOUN and other words |> Some details ``` Few rules that we can extract from above example: - Each line should start at least with one space. - First word should be a VERB and must be aligned to the right. - Second word should be a NOUN and must be aligned to the left. - If you can not find a good VERB / NOUN pair, don’t worry make it as understandable to the user as possible. - More details of each line can be provided by `|>` character which is serving as the first word when aligning the text Don’t forget you should also test your terminal output with colors and emojis off (`--no-colors --no-emojis`). ## Dim / Bright After comparing few terminals with different color schemes we would **recommend to avoid using dimmed text**. The difference from the rest of the text is very little in many terminal and color scheme combinations. Sometimes the difference is not even notable, therefore relying on it wouldn’t make much sense. **The bright text is much better supported** across terminals and color schemes. Most of the time the difference is perceived as if the bright text would be bold. ## Colors Humans are already conditioned by society to attach certain meaning to certain colors. While the meaning is not universal, a simple collection of colors is used to represent basic emotions. Colors that can be used in output - Red = error, danger, stop - Green = success, good - Yellow/Orange = proceed with caution, warning, in progress - Blue/Magenta = stability, calm While colors are nice, when command line is used by machines (in automation scripts) you want to remove the colors. There should be a global `--no-colors` option that would remove the colors. ## Special (Unicode) characters Most of the terminal have good support for Unicode characters and you should use them in your output by default. But always have a backup solution that is implemented only with ASCII characters and will be used when `--ascii` option is going to be passed in. Please make sure that you test your output also without Unicode characters More they showing all the different Unicode characters it is important to **establish common set of characters** that we use for certain situations. ## Emojis Emojis help channel emotions even better than text, colors and special characters. We recommend **keeping the set of emojis to a minimum**. This will enable each emoji to stand out more. As not everybody is happy about emojis we should provide an `--no-emojis` option to disable them. Please make sure that you test your output also without emojis. ## Tables All commands that are listing certain data can be implemented in some sort of a table. It’s important that each row of your output is a single ‘entry’ of data. Never output table borders. It’s noisy and a huge pain for parsing using other tools such as `grep`. Be mindful of the screen width. Only show a few columns by default with the table header, for more the table can be manipulated by the following options: - `--no-headers`: Show column headers by default but allow to hide them. - `--columns`: Comma-separated list of column names to add. - `--sort`: Allow sorting by column. Allow inverse and multi-column sort as well. ## Interactive output Interactive output was selected to be able to strike the balance between beginners and advance users. While the default output will target beginners it can, with a few key strokes, be changed into and advance introspection tool. ### Progress For longer running commands we should provide and overview the progress. This is shown best in `nix build` example: ``` $ nix build Started building my-project 1.2.3 Downloaded python3.8-poetry 1.2.3 in 5.3 seconds Downloaded python3.8-requests 1.2.3 in 5.3 seconds ------------------------------------------------------------------------ Press `v` to increase logs verbosity |> `?` to see other options ------------------------------------------------------------------------ Learn something new with every build... |> See last logs of a build with `nix log --last` command. ------------------------------------------------------------------------ Evaluated my-project 1.2.3 in 14.43 seconds Downloading [12 / 200] |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left Building [2 / 20] |> glibc 1.2.3 -> buildPhase: ------------------------------------------------------------------------ ``` ### Search Use a `fzf` like fuzzy search when there are multiple options to choose from. ``` $ nix init Initializing Nix project at `/path/to/here`. Select a template for you new project: |> py template#python-pip template#python-poetry [ Showing 2 templates from 1345 templates ] ``` ### Prompt In some situations we need to prompt the user and inform the user about what is going to happen. ``` $ nix build --option substitutors https://cache.example.org ------------------------------------------------------------------------ Warning! A security related question needs to be answered. ------------------------------------------------------------------------ The following substitutors will be used to in `my-project`: - https://cache.example.org Do you allow `my-project` to use above mentioned substitutors? [y/N] |> y ``` ## Verbosity There are many ways that you can control verbosity. Verbosity levels are: - `ERROR` (level 0) - `WARN` (level 1) - `NOTICE` (level 2) - `INFO` (level 3) - `TALKATIVE` (level 4) - `CHATTY` (level 5) - `DEBUG` (level 6) - `VOMIT` (level 7) The default level that the command starts is `ERROR`. The simplest way to increase the verbosity by stacking `-v` option (eg: `-vvv == level 3 == INFO`). There are also two shortcuts, `--debug` to run in `DEBUG` verbosity level and `--quiet` to run in `ERROR` verbosity level. * * * # Appendix 1: Commands naming exceptions `nix init` and `nix repl` are well established ## [Contributing](https://docharvest.github.io/docs/nix/development/contributing/) Contents nix Contributing Nix Contributing ## Add a release note `doc/manual/rl-next` contains release notes entries for all unreleased changes. User-visible changes should come with a release note. ### Add an entry Here's what a complete entry looks like. The file name is not incorporated in the document. ``` --- synopsis: Basically a title issues: 1234 prs: 1238 --- Here's one or more paragraphs that describe the change. - It's markdown - Add references to the manual using [links like this](@_at_docroot@/example.md) ``` Significant changes should add the following header, which moves them to the top. ``` significance: significant ``` See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog). ### Build process Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`. ## Branches - [`master`](https://github.com/NixOS/nix/commits/master) The main development branch. All changes are approved and merged here. When developing a change, create a branch based on the latest `master`. Maintainers try to [keep it in a release-worthy state](#reverting). - [`maintenance-*.*`](https://github.com/NixOS/nix/branches/all?query=maintenance) These branches are the subject of backports only, and are also [kept](#reverting) in a release-worthy state. See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md) - [`latest-release`](https://github.com/NixOS/nix/tree/latest-release) The latest patch release of the latest minor version. See [`maintainers/release-process.md`](https://github.com/NixOS/nix/blob/master/maintainers/release-process.md) - [`backport-*-to-*`](https://github.com/NixOS/nix/branches/all?query=backport) Generally branches created by the backport action. See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md) - [_other_](https://github.com/NixOS/nix/branches/all) Branches that do not conform to the above patterns should be feature branches. ## Reverting If a change turns out to be merged by mistake, or contain a regression, it may be reverted. A revert is not a rejection of the contribution, but merely part of an effective development process. It makes sure that development keeps running smoothly, with minimal uncertainty, and less overhead. If maintainers have to worry too much about avoiding reverts, they would not be able to merge as much. By embracing reverts as a good part of the development process, everyone wins. However, taking a step back may be frustrating, so maintainers will be extra supportive on the next try. ## [C++ style guide](https://docharvest.github.io/docs/nix/development/cxx/) Contents nix C++ style guide Nix C++ style guide Some miscellaneous notes on how we write C++. Formatting we hope to eventually normalize automatically, so this section is free to just discuss higher-level concerns. ## The `*-impl.hh` pattern Let's start with some background info first. Headers, are supposed to contain declarations, not definitions. This allows us to change a definition without changing the declaration, and have a very small rebuild during development. Templates, however, need to be specialized to use-sites. Absent fancier techniques, templates require that the definition, not just mere declaration, must be available at use-sites in order to make that specialization on the fly as part of compiling those use-sites. Making definitions available like that means putting them in headers, but that is unfortunately means we get all the extra rebuilds we want to avoid by just putting declarations there as described above. The `*-impl.hh` pattern is a ham-fisted partial solution to this problem. It constitutes: - Declaring items only in the main `foo.hh`, including templates. - Putting template definitions in a companion `foo-impl.hh` header. Most C++ developers would accompany this by having `foo.hh` include `foo-impl.hh`, to ensure any file getting the template declarations also got the template definitions. But we've found not doing this has some benefits and fewer than imagined downsides. The fact remains that headers are rarely as minimal as they could be; there is often code that needs declarations from the headers but not the templates within them. With our pattern where `foo.hh` doesn't include `foo-impl.hh`, that means they can just include `foo.hh` Code that needs both just includes `foo.hh` and `foo-impl.hh`. This does make linking error possible where something forgets to include `foo-impl.hh` that needs it, but those are build-time only as easy to fix. ## [Data Modeling Guidelines](https://docharvest.github.io/docs/nix/development/data-modeling/) Contents nix Data Modeling Guidelines Nix Data Modeling Guidelines Nix consumes and produces JSON and attribute sets in a variety of contexts. These guidelines ensure consistent practices for our interfaces, for ease of use, and so that experience in one part carries over to another. For these guidelines, we will use JSON terminology, but they apply equally well to new attribute set interfaces (primops, etc.). Note that these are guidelines first and foremost. Exceptions include: - Feature testing: e.g., it is OK to do `builtins?frobnicate`. - Compatibility: we generally do not change stable interfaces just to make them comply. New replacements can be added with care. ## Extensibility The schema of JSON input and output should allow for backwards compatible extension. This section explains how to achieve this. Two definitions are helpful here, because while JSON only defines one "key-value" object type, we use it to cover two use cases: - **dictionary**: a map from names to value that all have the same type. In C++ this would be a `std::map` with string keys. - **record**: a fixed set of attributes each with their own type. In C++, this would be represented by a `struct`. It is best not to mix these use cases, as that may lead to incompatibilities when the schema changes. For example, adding a record field to a dictionary breaks consumers that assume all JSON object fields to have the same meaning and type, and dictionary items with a colliding name can not be represented anymore. This leads to the following guidelines: - The top-level (root) value must be a record. Otherwise, one can not change the structure of a command's output. - The value of a dictionary item must be a record. Otherwise, the item type can not be extended. - List items should be records. Otherwise, one can not change the structure of the list items. If the order of the items does not matter, and each item has a unique key that is a string, consider representing the list as a dictionary instead. If the order of the items needs to be preserved, return a list of records. - Streaming JSON should return records. An example of a streaming JSON format is [JSON lines](https://jsonlines.org/), where each line represents a JSON value. These JSON values can be considered top-level values or list items, and they must be records. ### Examples This is bad, because all keys must be assumed to be store types: ``` { "local": { ... }, "remote": { ... }, "http": { ... } } ``` This is good, because the it is extensible at the root, and is somewhat self-documenting: ``` { "storeTypes": { "local": { ... }, ... }, "pluginSupport": true } ``` While the dictionary of store types seems like a very complete response at first, a use case may arise that warrants returning additional information. For example, the presence of plugin support may be crucial information for a client to proceed when their desired store type is missing. The following representation is bad because it is not extensible: ``` { "outputs": [ "out" "bin" ] } ``` However, simply converting everything to records is not enough, because the order of outputs must be preserved: ``` { "outputs": { "bin": {}, "out": {} } } ``` The first item is the default output. Deriving this information from the outputs ordering is not great, but this is how Nix currently happens to work. While it is possible for a JSON parser to preserve the order of fields, we can not rely on this capability to be present in all JSON libraries. This representation is extensible and preserves the ordering: ``` { "outputs": [ { "outputName": "out" }, { "outputName": "bin" } ] } ``` ## Self-describing values As described in the previous section, it's crucial that schemas can be extended with new fields without breaking compatibility. However, that should _not_ mean we use the presence/absence of fields to indicate optional information _within_ a version of the schema. Instead, always include the field, and use `null` to indicate the "nothing" case. ### Examples Here are two JSON objects: ``` { "foo": {} } ``` ``` { "foo": {}, "bar": {} } ``` Since they differ in which fields they contain, they should _not_ both be valid values of the same schema. At most, they can match two different schemas where the second (with `foo` and `bar`) is considered a newer version of the first (with just `foo`). Within each version, all fields are mandatory (always `foo`, and always `foo` and `bar`). Only _between_ each version, `bar` gets added as a new mandatory field. Here are another two JSON objects: ``` { "foo": null } ``` ``` { "foo": { "bar": 1 } } ``` Since they both contain a `foo` field, they could be valid values of the same schema. The schema would have `foo` has an optional field, which is either `null` or an object where `bar` is an integer. ## [Debugging Nix](https://docharvest.github.io/docs/nix/development/debugging/) Contents nix Debugging Nix Nix Debugging Nix This section shows how to build and debug Nix with debug symbols enabled. Additionally, see [Testing Nix](./testing.md) for further instructions on how to debug Nix in the context of a unit test or functional test. ## Building Nix with Debug Symbols In the development shell, `mesonBuildType` is set automatically to `debugoptimized`. This builds Nix with debug symbols, which are essential for effective debugging. It is also possible to build without optimization for faster build: ``` [nix-shell]$ NIX_HARDENING_ENABLE=$(printLines $NIX_HARDENING_ENABLE | grep -v fortify) [nix-shell]$ export mesonBuildType=debug ``` (The first line is needed because `fortify` hardening requires at least some optimization.) ## Building Nix with sanitizers Nix can be built with [Address](https://clang.llvm.org/docs/AddressSanitizer.html) and [UB](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) sanitizers using LLVM or GCC. This is useful when debugging memory corruption issues. ``` [nix-shell]$ export mesonBuildType=debugoptimized [nix-shell]$ appendToVar mesonFlags "-Dlibexpr:gc=disabled" # Disable Boehm [nix-shell]$ appendToVar mesonFlags "-Db_sanitize=address,undefined" ``` ## Debugging the Nix Binary Obtain your preferred debugger within the development shell: ``` [nix-shell]$ nix-shell -p gdb ``` On macOS, use `lldb`: ``` [nix-shell]$ nix-shell -p lldb ``` ### Launching the Debugger To debug the Nix binary, run: ``` [nix-shell]$ gdb --args ../outputs/out/bin/nix ``` On macOS, use `lldb`: ``` [nix-shell]$ lldb -- ../outputs/out/bin/nix ``` ### Using the Debugger Inside the debugger, you can set breakpoints, run the program, and inspect variables. ``` (gdb) break main (gdb) run ``` Refer to the [GDB Documentation](https://www.gnu.org/software/gdb/documentation/) for comprehensive usage instructions. On macOS, use `lldb`: ``` (lldb) breakpoint set --name main (lldb) process launch -- ``` Refer to the [LLDB Tutorial](https://lldb.llvm.org/use/tutorial.html) for comprehensive usage instructions. ## [Contributing documentation](https://docharvest.github.io/docs/nix/development/documentation/) Contents nix Contributing documentation Nix Contributing documentation Improvements to documentation are very much appreciated, and a good way to start out with contributing to Nix. This is how you can help: - Address [open issues with documentation](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation) - Review [pull requests concerning documentation](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+label%3Adocumentation) Incremental refactorings of the documentation build setup to make it faster or easier to understand and maintain are also welcome. ## Building the manual Build the manual from scratch: ``` nix-build -E '(import ./.).packages.${builtins.currentSystem}.nix.doc' ``` or ``` nix build .#nix-manual ``` and open `./result/share/doc/nix/manual/index.html`. To build the manual incrementally, [enter the development shell](./building.md) and configure with `doc-gen` enabled: **If using interactive `nix develop`:** ``` $ nix develop $ mesonFlags="$mesonFlags -Ddoc-gen=true" mesonConfigurePhase ``` **If using direnv:** ``` $ direnv allow $ bash -c 'source $stdenv/setup && mesonFlags="$mesonFlags -Ddoc-gen=true" mesonConfigurePhase' ``` Then build the manual: ``` $ cd build $ meson compile manual ``` The HTML manual will be generated at `build/src/nix-manual/manual/index.html`. ## Style guide The goal of this style guide is to make it such that - The manual is easy to search and skim for relevant information - Documentation sources are easy to edit - Changes to documentation are easy to review You will notice that this is not implemented consistently yet. Please follow the guide when making additions or changes to existing documentation. Do not make sweeping changes, unless they are programmatic and can be validated easily. ### Language This manual is [reference documentation](https://diataxis.fr/reference/). The typical usage pattern is to look up isolated pieces of information. It should therefore aim to be correct, consistent, complete, and easy to navigate at a glance. - Aim for clarity and brevity. Please take the time to read the [plain language guidelines](https://www.plainlanguage.gov/guidelines/) for details. - Describe the subject factually. In particular, do not make value judgements or recommendations. Check the code or add tests if in doubt. - Provide complete, minimal examples, and explain them. Readers should be able to try examples verbatim and get the same results as shown in the manual. Always describe in words what a given example does. Non-trivial examples may need additional explanation, especially if they use concepts from outside the given context. - Always explain code examples in the text. Use comments in code samples very sparingly, for instance to highlight a particular aspect. Readers tend to glance over large amounts of code when scanning for information. Especially beginners will likely find reading more complex-looking code strenuous and may therefore avoid it altogether. If a code sample appears to require a lot of inline explanation, consider replacing it with a simpler one. If that's not possible, break the example down into multiple parts, explain them separately, and then show the combined result at the end. This should be a last resort, as that would amount to writing a [tutorial](https://diataxis.fr/tutorials/) on the given subject. - Use British English. This is a somewhat arbitrary choice to force consistency, and accounts for the fact that a majority of Nix users and developers are from Europe. ### Links and anchors Reference documentation must be readable in arbitrary order. Readers cannot be expected to have any particular prerequisite knowledge about Nix. While the table of contents can provide guidance and full-text search can help, they are most likely to find what they need by following sensible cross-references. - Link to technical terms When mentioning Nix-specific concepts, commands, options, settings, etc., link to appropriate documentation. Also link to external tools or concepts, especially if their meaning may be ambiguous. You may also want to link to definitions of less common technical terms. Then readers won't have to actively search for definitions and are more likely to discover relevant information on their own. > **Note** > > `man` and `--help` pages don't display links. Use appropriate link texts such that readers of terminal output can infer search terms. - Do not break existing URLs between releases. There are countless links in the wild pointing to old versions of the manual. We want people to find up-to-date documentation when following popular advice. - When moving files, update [redirects on nixos.org](https://github.com/NixOS/nixos-homepage/blob/master/netlify.toml). This is especially important when moving information out of the Nix manual to other resources. - When changing anchors, update [client-side redirects](https://github.com/NixOS/nix/blob/master/doc/manual/redirects.js) The current setup is cumbersome, and help making better automation is appreciated. The build checks for broken internal links with. This happens late in the process, so [building the whole manual](#building-the-manual) is not suitable for iterating quickly. [`mdbook-linkcheck`](https://github.com/Michael-F-Bryan/mdbook-linkcheck) does not implement checking [URI fragments](https://en.wikipedia.org/wiki/URI_fragment) yet. ### Markdown conventions The manual is written in markdown, and rendered with [mdBook](https://github.com/rust-lang/mdBook) for the web and with [lowdown](https://github.com/kristapsdz/lowdown) for `man` pages and `--help` output. For supported markdown features, refer to: - [mdBook documentation](https://rust-lang.github.io/mdBook/format/markdown.html) - [lowdown documentation](https://kristaps.bsd.lv/lowdown/) Please observe these guidelines to ease reviews: - Write one sentence per line. This makes long sentences immediately visible, and makes it easier to review changes and make direct suggestions. - Use reference links – sparingly – to ease source readability. Put definitions close to their first use. Example: ``` A [store object] contains a [file system object] and [references] to other store objects. [store object]: @docroot@/store/store-object.md [file system object]: @docroot@/architecture/file-system-object.md [references]: @docroot@/glossary.md#gloss-reference ``` - Use admonitions of the following form: ``` > **Note** > > This is a note. ``` Highlight examples as such: ``` > **Example** > > ```console > $ nix --version > ``` ``` Highlight syntax definitions as such, using [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) notation: ``` > **Syntax** > > *attribute-set* = `{` [ *attribute-name* `=` *expression* `;` ... ] `}` ``` ### The `@docroot@` variable `@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own. If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/source` directory. If the `@docroot@` literal appears in an error message from the [`mdbook-linkcheck`](https://github.com/Michael-F-Bryan/mdbook-linkcheck) tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it. See existing `@docroot@` logic in the \[Makefile for the manual\]. Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`. ## API documentation [Doxygen API documentation](https://hydra.nixos.org/job/nix/master/internal-api-docs/latest/download-by-type/doc/internal-api-docs) is available online. You can also build and view it yourself: ``` $ nix build .#hydraJobs.internal-api-docs $ xdg-open ./result/share/doc/nix/internal-api/html/index.html ``` or inside `nix-shell` or `nix develop`: ``` $ configurePhase $ ninja src/internal-api-docs/html $ xdg-open src/internal-api-docs/html/index.html ``` ## C API documentation Note that the C API is not yet stable. [C API documentation](https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs) is available online. You can also build and view it yourself: ``` $ nix build .#hydraJobs.external-api-docs $ xdg-open ./result/share/doc/nix/external-api/html/index.html ``` or inside `nix-shell` or `nix develop`: ``` $ configurePhase $ ninja src/external-api-docs/html $ xdg-open src/external-api-docs/html/index.html ``` If you use direnv, or otherwise want to run `configurePhase` in a transient shell, use: ``` nix-shell -A devShells.x86_64-linux.native-clangStdenv --command 'appendToVar mesonFlags "-Ddoc-gen=true"; mesonConfigurePhase' ``` ## [Experimental Features](https://docharvest.github.io/docs/nix/development/experimental-features/) Contents nix Experimental Features Nix Experimental Features This section describes the notion of _experimental features_, and how it fits into the big picture of the development of Nix. # What are experimental features? Experimental features are considered unstable, which means that they can be changed or removed at any time. Users must explicitly enable them by toggling the associated [experimental feature flags](@docroot@/command-ref/conf-file.md#conf-experimental-features). This allows accessing unstable functionality without unwittingly relying on it. Experimental feature flags were first introduced in [Nix 2.4](@docroot@/release-notes/rl-2.4.md). Before that, Nix did have experimental features, but they were not guarded by flags and were merely documented as unstable. This was a source of confusion and controversy. # When should a new feature be marked experimental? A change in the Nix codebase should be guarded by an experimental feature flag if it is considered likely to be reverted or adapted in a backwards-incompatible manner after gathering more experience with it in practice. Examples: - Changes to the Nix language, such as new built-ins, syntactic or semantic changes, etc. - Changes to the command-line interface # Lifecycle of an experimental feature Experimental features have to be treated on a case-by-case basis. However, the standard workflow for an experimental feature is as follows: - A new feature is implemented in a _pull request_ - It is guarded by an experimental feature flag that is disabled by default - The pull request is merged, the _experimental_ feature ends up in a release - Using the feature requires explicitly enabling it, signifying awareness of the potential risks - Being experimental, the feature can still be changed arbitrarily - The feature can be _removed_ - The associated experimental feature flag is also removed - The feature can be declared _stable_ - The associated experimental feature flag is removed - There should be enough evidence of users having tried the feature, such as feedback, fixed bugs, demonstrations of how it is put to use - Maintainers must feel confident that: - The feature is designed and implemented sensibly, that it is fit for purpose - Potential interactions are well-understood - Stabilising the feature will not incur an outsized maintenance burden in the future The following diagram illustrates the process: ``` .------. | idea | '------' | discussion, design, implementation | | .-------. | | | v v | .--------------. review | pull request | | '--------------' | | ^ | | | | '-------' .---' '----. | | merge user feedback, | (breaking) changes | | '---. .----' | | v | +--------------+ .---| experimental |----. | +--------------+ | | | decision to stabilise decision against | keeping the feature | | v v +--------+ +---------+ | stable | | removed | +--------+ +---------+ ``` # Relation to the RFC process Experimental features and [RFCs](https://github.com/NixOS/rfcs/) both allow approaching substantial changes while minimizing the risk. However they serve different purposes: - An experimental feature enables developers to iterate on and deliver a new idea without committing to it or requiring a costly long-running fork. It is primarily an issue of _implementation_, targeting Nix developers and early testers. - The goal of an RFC is to make explicit all the implications of a change: Explain why it is wanted, which new use-cases it enables, which interface changes it requires, etc. It is primarily an issue of _design_ and _communication_, targeting the broader community. This means that experimental features and RFCs are orthogonal mechanisms, and can be used independently or together as needed. # Currently available experimental features {{#include ./experimental-feature-descriptions.md}} ## [Static Analysis](https://docharvest.github.io/docs/nix/development/static-analysis/) Contents nix Static Analysis Nix Static Analysis Nix uses [clang-tidy](https://clang.llvm.org/extra/clang-tidy/) for static analysis of C++ code. This helps catch bugs, enforce coding standards, and maintain code quality. ## Running clang-tidy locally To run clang-tidy on the entire codebase in the development shell: ``` $ nix develop .#native-clangStdenv $ configurePhase $ meson compile -C build clang-tidy ``` This will analyze all C++ source files and report any warnings. To automatically apply fixes for certain warnings: ``` $ meson compile -C build clang-tidy-fix ``` > **Warning** > > Review the changes before committing, as automatic fixes may not always be correct. ## CI integration clang-tidy runs automatically on every pull request via GitHub Actions. The CI job builds `.#hydraJobs.clangTidy.x86_64-linux` which: 1. Builds all components with debug mode (for faster compilation) 2. Runs clang-tidy on each component 3. Fails if any warnings are found (warnings are treated as errors) ## Configuration The clang-tidy configuration is in `.clang-tidy` at the repository root (symlinked from `nix-meson-build-support/common/clang-tidy/.clang-tidy`). ### Suppressing warnings If a warning is a false positive, you can suppress it in several ways: 1. **Inline suppression** (preferred for specific cases): ``` // NOLINTBEGIN(bugprone-some-check) ... code ... // NOLINTEND(bugprone-some-check) ``` Or for a single line: ``` int x = something(); // NOLINT(bugprone-some-check) ``` 2. **Configuration file** (for project-wide suppression): Add the check to the disabled list in `.clang-tidy`: ``` Checks: - -bugprone-some-check # Reason for disabling ``` 3. **Check options** (for configuring check behavior): ``` CheckOptions: bugprone-reserved-identifier.AllowedIdentifiers: '__some_identifier' ``` ### Adding new checks To enable additional checks: 1. Edit `nix-meson-build-support/common/clang-tidy/.clang-tidy` 2. Add the check to the `Checks` list 3. Run clang-tidy locally to see the impact 4. Fix any new warnings or disable specific sub-checks if needed ## Custom clang-tidy plugin The Nix project includes infrastructure for custom clang-tidy checks in `src/clang-tidy-plugin/`. These checks can enforce Nix-specific coding patterns that aren't covered by standard clang-tidy checks. To add a new custom check: 1. Add the check implementation in `src/clang-tidy-plugin/` 2. Register it in `nix-clang-tidy-checks.cc` 3. Enable it in `.clang-tidy` with the `nix-` prefix ## [Running tests](https://docharvest.github.io/docs/nix/development/testing/) Contents nix Running tests Nix Running tests ## Coverage analysis A [coverage analysis report](https://hydra.nixos.org/job/nix/master/coverage/latest/download-by-type/report/coverage) is available online You can build it yourself: ``` # nix build .#hydraJobs.coverage # xdg-open ./result/index.html ``` [Extensive records of build metrics](https://hydra.nixos.org/job/nix/master/coverage#tabs-charts), such as test coverage over time, are also available online. ## Unit-tests The unit tests are defined using the [googletest](https://google.github.io/googletest/) and [rapidcheck](https://github.com/emil-e/rapidcheck) frameworks. ### Source and header layout > An example of some files, demonstrating much of what is described below > > ``` > src > ├── libexpr > │ ├── meson.build > │ ├── include/nix/expr/value/context.hh > │ ├── value/context.cc > │ … > │ > ├── libutil-tests > │ ├── meson.build > │ … > │ ├── data > │ │ ├── git/tree.txt > │ … > │ > ├── libexpr-test-support > │ ├── meson.build > │ ├── include/nix/expr > │ │ ├── meson.build > │ │ └── tests > │ │ ├── value/context.hh > │ │ … > │ ├── tests > │ ├── value/context.cc > │ … > │ > ├── libexpr-tests > │ ├── meson.build > │ ├── value/context.cc > │ … > … > ``` The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`. Given an interface (header) and implementation pair in the original library, say, `src/libexpr/include/nix/expr/value/context.hh` and `src/libexpr/value/context.cc`, we write tests for it in `src/libexpr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/libexpr-test-support/include/nix/expr/tests/value/context.hh` and `src/libexpr-test-support/tests/value/context.cc`. Data for unit tests is stored in a `data` subdir of the directory for each unit test executable. For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/libstore-tests/data`. The path to the `src/${library_name_without-nix}-test/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`. Note that each executable only gets the data for its tests. The unit test libraries are in `src/${library_name_without-nix}-test-support`. All headers are in a `tests` subdirectory so they are included with `#include "nix/tests/"`. The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing. But organizing the tests this way has one big benefit: there is no risk of any build-system wildcards for the library accidentally picking up test code that should not built and installed as part of the library. ### Running tests You can run the whole testsuite with `meson test` from the Meson build directory, or the tests for a specific component with `meson test nix-store-tests`. The environment variables that Google Test accepts are also worth knowing: 1. [`GTEST_FILTER`](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) This is used for finer-grained filtering of which tests to run. 2. [`GTEST_BRIEF`](https://google.github.io/googletest/advanced.html#suppressing-test-passes) This is used to avoid logging passing tests. 3. [`GTEST_BREAK_ON_FAILURE`](https://google.github.io/googletest/advanced.html#turning-assertion-failures-into-break-points) This is used to create a debugger breakpoint when an assertion failure occurs. Putting the first two together, one might run ``` GTEST_BRIEF=1 GTEST_FILTER='ErrorTraceTest.*' meson test nix-expr-tests -v ``` for short but comprensive output. ### Debugging tests For debugging, it is useful to combine the third option above with Meson's [`--gdb`](https://mesonbuild.com/Unit-tests.html#other-test-options) flag: ``` GTEST_BRIEF=1 GTEST_FILTER='Group.my-failing-test' meson test nix-expr-tests --gdb ``` This will: 1. Run the unit test with GDB 2. Run just `Group.my-failing-test` 3. Stop the program when the test fails, allowing the user to then issue arbitrary commands to GDB. ### Characterisation testing { #characterisation-testing-unit } See [functional characterisation testing](#characterisation-testing-functional) for a broader discussion of characterisation testing. Like with the functional characterisation, `_NIX_TEST_ACCEPT=1` is also used. For example: ``` $ _NIX_TEST_ACCEPT=1 meson test nix-store-tests -v ... [ SKIPPED ] WorkerProtoTest.string_read [ SKIPPED ] WorkerProtoTest.string_write [ SKIPPED ] WorkerProtoTest.storePath_read [ SKIPPED ] WorkerProtoTest.storePath_write ... ``` will regenerate the "golden master" expected result for the `libnixstore` characterisation tests. The characterisation tests will mark themselves "skipped" since they regenerated the expected result instead of actually testing anything. ### JSON Schema testing In `doc/manual/source/protocols/json/` we have a number of manual pages generated from [JSON Schema](https://json-schema.org/). That JSON schema is tested against the JSON file test data used in [characterisation tests](#characterisation-testing-unit) for JSON (de)serialization, in `src/json-schema-checks`. Between the JSON (de)serialization testing, and this testing of the same data against the schema, we make sure that the manual, the implementation, and a machine-readable schema are are all in sync. ### Unit test support libraries There are headers and code which are not just used to test the library in question, but also downstream libraries. For example, we do [property testing](https://en.wikipedia.org/wiki/Property_testing) with the [rapidcheck](https://github.com/emil-e/rapidcheck) library. This requires writing `Arbitrary` "instances", which are used to describe how to generate values of a given type for the sake of running property tests. Because types contain other types, `Arbitrary` "instances" for some type are not just useful for testing that type, but also any other type that contains it. Downstream types frequently contain upstream types, so it is very important that we share arbitrary instances so that downstream libraries' property tests can also use them. It is important that these testing libraries don't contain any actual tests themselves. On some platforms they would be run as part of every test executable that uses them, which is redundant. On other platforms they wouldn't be run at all. ## Functional tests The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/meson.build`. Each test is a bash script. Functional tests are run during `installCheck` in the `nix` package build, as well as separately from the build, in VM tests. ### Running the whole test suite The whole test suite (functional and unit tests) can be run with: ``` $ checkPhase ``` ### Grouping tests Sometimes it is useful to group related tests so they can be easily run together without running the entire test suite. Each test group is in a subdirectory of `tests`. For example, `tests/functional/ca/meson.build` defines a `ca` test group for content-addressing derivation outputs. That test group can be run like this: ``` $ meson test --suite ca ninja: Entering directory `/home/jcericson/src/nix/master/build' ninja: no work to do. [1-20/20] 🌑 nix-functional-tests:ca / ca/why-depends 1/20 nix-functional-tests:ca / ca/nix-run OK 0.16s [2-20/20] 🌒 nix-functional-tests:ca / ca/why-depends 2/20 nix-functional-tests:ca / ca/import-derivation OK 0.17s ``` ### Running individual tests Individual tests can be run with `meson`: ``` $ meson test --verbose ${testName} ninja: Entering directory `/home/jcericson/src/nix/master/build' ninja: no work to do. 1/1 nix-functional-tests:main / ${testName} OK 0.41s Ok: 1 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 0 Timeout: 0 Full log written to /home/jcericson/src/nix/master/build/meson-logs/testlog.txt ``` The `--verbose` flag will make Meson also show the console output of each test for easier debugging. The test script will then be traced with `set -x` and the output displayed as it happens, regardless of whether the test succeeds or fails. Tests can be also run directly without `meson`: ``` $ TEST_NAME=${testName} NIX_REMOTE='' PS4='+(${BASH_SOURCE[0]-$0}:$LINENO) tests/functional/${testName}.sh +(${testName}.sh:1) foo output from foo +(${testName}.sh:2) bar output from bar ... ``` ### Debugging failing functional tests When a functional test fails, it usually does so somewhere in the middle of the script. To figure out what's wrong, it is convenient to run the test regularly up to the failing `nix` command, and then run that command with a debugger like GDB. For example, if the script looks like: ``` foo nix blah blub bar ``` edit it like so: ``` foo -nix blah blub +gdb --args nix blah blub bar ``` Then, running the test with [`--interactive`](https://mesonbuild.com/Unit-tests.html#other-test-options) will prevent Meson from hijacking the terminal so you can drop you into GDB once the script reaches that point: ``` $ meson test ${testName} --interactive ... + gdb blash blub GNU gdb (GDB) 12.1 ... (gdb) ``` One can debug the Nix invocation in all the usual ways. For example, enter `run` to start the Nix invocation. ### Characterisation testing { #characterisation-testing-functional } Occasionally, Nix utilizes a technique called [Characterisation Testing](https://en.wikipedia.org/wiki/Characterization_test) as part of the functional tests. This technique is to include the exact output/behavior of a former version of Nix in a test in order to check that Nix continues to produce the same behavior going forward. For example, this technique is used for the language tests, to check both the printed final value if evaluation was successful, and any errors and warnings encountered. It is frequently useful to regenerate the expected output. To do that, rerun the failed test(s) with `_NIX_TEST_ACCEPT=1`. For example: ``` _NIX_TEST_ACCEPT=1 meson test lang ``` This convention is shared with the [characterisation unit tests](#characterisation-testing-unit) too. An interesting situation to document is the case when these tests are "overfitted". The language tests are, again, an example of this. The expected successful output of evaluation is supposed to be highly stable – we do not intend to make breaking changes to (the stable parts of) the Nix language. However, the errors and warnings during evaluation (successful or not) are not stable in this way. We are free to change how they are displayed at any time. It may be surprising that we would test non-normative behavior like diagnostic outputs. Diagnostic outputs are indeed not a stable interface, but they still are important to users. By recording the expected output, the test suite guards against accidental changes, and ensure the _result_ (not just the code that implements it) of the diagnostic code paths are under code review. Regressions are caught, and improvements always show up in code review. To ensure that characterisation testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`. ### Running functional tests on NixOS We run the functional tests not just in the build, but also in VM tests. This helps us ensure that Nix works correctly on NixOS, and environments that have similar characteristics that are hard to reproduce in a build environment. These can be run with: ``` nix build .#hydraJobs.tests.functional_user ``` Generally, this build is sufficient, but in nightly or CI we also test the attributes `functional_root` and `functional_trusted`, in which the test suite is run with different levels of authorization. ## Integration tests The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute. These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup. Because these tests are expensive and require more than what the standard github-actions setup provides, most of them only run on the master branch (on [https://hydra.nixos.org/jobset/nix/master](https://hydra.nixos.org/jobset/nix/master)). You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`. ## Installer tests GitHub Actions CI in the Nix repository also tests the installer on PRs. It does not require additional setup and utilises [GHA Artifacts](https://docs.github.com/en/actions/tutorials/store-and-share-data) and can be run in any Nix repository fork. - The `tests` job generates installers for the platforms below and uploads them as an artifact: - `x86_64-linux` - `aarch64-darwin` - The `installer_test` job (which runs on Linux and macOS) will try to install Nix with the cached installer and run a trivial Nix command. - Both the scripted installer and the [standalone Rust-based installer](https://github.com/NixOS/nix-installer) are tested. You can generate the installer tarball and script manually by running `nix build .#hydraJobs.installerScriptForGHA.`. ## Working on documentation ### Using the CI-generated installer for manual testing After the CI run completes, you can check the output to extract the installer artifact: 1. Click into the detailed view of the CI run. 2. Scroll down to `Artifacts` section. 3. Download the corresponding installer artifact (`installer-darwin` for `aarch64-darwin` and `installer-linux` for `x86_64-linux`). 4. Unpack the downloaded `.zip` artifact. 5. To generate an install command, plug the path to the unpacked artifact into this template: ``` sh /install --tarball-url-prefix file:// ``` ## [Glossary](https://docharvest.github.io/docs/nix/glossary/) Contents nix Glossary Nix Glossary - \[`∅`\]{#gloss-empty-set} The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile. - \[`ε`\]{#gloss-epsilon} The epsilon symbol. In the context of a package, this means the version is empty. More precisely, the derivation does not have a version attribute. - [base directory](#gloss-base-directory){#gloss-base-directory} The location from which relative paths are resolved. - For expressions in a file, the base directory is the directory containing that file. This is analogous to the directory of a [base URL](https://datatracker.ietf.org/doc/html/rfc1808#section-3.3). - For expressions written in command line arguments with [`--expr`](@docroot@/command-ref/opt-common.html#opt-expr), the base directory is the current working directory. - \[binary cache\]{#gloss-binary-cache} A _binary cache_ is a Nix store which uses a different format: its metadata and signatures are kept in `.narinfo` files rather than in a [Nix database](#gloss-nix-database). This different format simplifies serving store objects over the network, but cannot host builds. Examples of binary caches include S3 buckets and the [NixOS binary cache](https://cache.nixos.org). - \[build system\]{#gloss-build-system} Generic term for software that facilitates the building of software by automating the invocation of compilers, linkers, and other tools. Nix can be used as a generic build system. It has no knowledge of any particular programming language or toolchain. These details are specified in [derivation expressions](#gloss-derivation-expression). - [closure](#gloss-closure){#gloss-closure} The closure of a store path is the set of store paths that are directly or indirectly “reachable” from that store path; that is, it’s the closure of the path under the _references_ relation. For a package, the closure of its derivation is equivalent to the build-time dependencies, while the closure of its [output path](#gloss-output-path) is equivalent to its runtime dependencies. For correct deployment it is necessary to deploy whole closures, since otherwise at runtime files could be missing. The command `nix-store --query --requisites` prints out closures of store paths. As an example, if the [store object](#gloss-store-object) at path `P` contains a [reference](#gloss-reference) to a store object at path `Q`, then `Q` is in the closure of `P`. Further, if `Q` references `R` then `R` is also in the closure of `P`. See [References](@docroot@/store/store-object.md#references) for details. - \[content address\]{#gloss-content-address} A [_content address_](https://en.wikipedia.org/wiki/Content-addressable_storage) is a secure way to reference immutable data. The reference is calculated directly from the content of the data being referenced, which means the reference is [_tamper proof_](https://en.wikipedia.org/wiki/Tamperproofing) --- variations of the data should always calculate to distinct content addresses. For how Nix uses content addresses, see: - [Content-Addressing File System Objects](@docroot@/store/file-system-object/content-address.md) - [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md) - [content-addressing derivation](#gloss-content-addressing-derivation) Software Heritage's writing on [_Intrinsic and Extrinsic identifiers_](https://www.softwareheritage.org/2020/07/09/intrinsic-vs-extrinsic-identifiers) is also a good introduction to the value of content-addressing over other referencing schemes. Besides content addressing, the Nix store also uses [input addressing](#gloss-input-addressed-store-object). - \[content-addressed storage\]{#gloss-content-addressed-store} The industry term for storage and retrieval systems using [content addressing](#gloss-content-address). A Nix store also has [input addressing](#gloss-input-addressed-store-object), and metadata. - \[content-addressed store object\]{#gloss-content-addressed-store-object} A [store object](#gloss-store-object) which is [content-addressed](#gloss-content-address), i.e. whose [store path](#gloss-store-path) is determined by its contents. This includes derivations, the outputs of [content-addressing derivations](#gloss-content-addressing-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation). See [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md) for details. - \[content-addressing derivation\]{#gloss-content-addressing-derivation} A derivation which has the [`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed) attribute set to `true`. - [derivation](#gloss-derivation){#gloss-derivation} A derivation can be thought of as a [pure function](https://en.wikipedia.org/wiki/Pure_function) that produces new [store objects](#gloss-store-object) from existing store objects. Derivations are implemented as [operating system processes that run in a sandbox](@docroot@/store/building.md). This sandbox by default only allows reading from store objects specified as inputs, and only allows writing to designated [outputs](#gloss-output) to be [captured as store objects](@docroot@/store/building.md#processing-outputs). A derivation is typically specified as a [derivation expression](#gloss-derivation-expression) in the [Nix language](./language/index.md), and [instantiated](#gloss-instantiate) to a [store derivation](#gloss-store-derivation). There are multiple ways of obtaining store objects from store derivations, collectively called [realisation](#gloss-realise). - [derivation expression](#gloss-derivation-expression){#gloss-derivation-expression} A description of a [store derivation](#gloss-store-derivation) using the [`derivation` primitive](./language/derivations.md) in the [Nix language](./language/index.md). - [derivation path](#gloss-derivation-path){#gloss-derivation-path} A [store path](#gloss-store-path) which uniquely identifies a [store derivation](#gloss-store-derivation). See [Referencing Store Derivations](@docroot@/store/derivation/index.md#derivation-path) for details. Not to be confused with \[deriving path\]. - \[deriver\]{#gloss-deriver} The [store derivation](#gloss-store-derivation) that produced an [output path](#gloss-output-path). The deriver for an output path can be queried with the `--deriver` option to [`nix-store --query`](@docroot@/command-ref/nix-store/query.md). - \[deriving path\]{#gloss-deriving-path} Deriving paths are a way to refer to [store objects](#gloss-store-object) that might not yet be [realised](#gloss-realise). See [Deriving Path](./store/derivation/index.md#deriving-path) for details. Not to be confused with [derivation path](#gloss-derivation-path). - \[directed acyclic graph\]{#gloss-directed-acyclic-graph} A [directed acyclic graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAG) is graph whose edges are given a direction ("a to b" is not the same edge as "b to a"), and for which no possible path (created by joining together edges) forms a cycle. DAGs are very important to Nix. In particular, the non-self-[references](#gloss-reference) of [store object](#gloss-store-object) form a cycle. - \[experimental feature\]{#gloss-experimental-feature} Not yet stabilized functionality guarded by named experimental feature flags. These flags are enabled or disabled with the [`experimental-features`](./command-ref/conf-file.html#conf-experimental-features) setting. See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/development/experimental-features.md). - [file system object](#gloss-file-system-object){#gloss-file-system-object} The Nix data model for representing simplified file system data. See [File System Object](@docroot@/store/file-system-object.md) for details. - \[fixed-output derivation\]{#gloss-fixed-output-derivation} (FOD) A [store derivation](#gloss-store-derivation) where a cryptographic hash of the [output](#gloss-output) is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network. - \[IFD\]{#gloss-ifd} [Import From Derivation](./language/import-from-derivation.md) - \[impure derivation\]{#gloss-impure-derivation} [An experimental feature](@docroot@/development/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure, so that they are always rebuilt, and their outputs not reused by subsequent calls to realise them. - \[input-addressed store object\]{#gloss-input-addressed-store-object} A store object produced by building a non-[content-addressed](#gloss-content-addressing-derivation), non-[fixed-output](#gloss-fixed-output-derivation) derivation. See [input-addressing derivation outputs](store/derivation/outputs/input-address.md) for details. - \[installable\]{#gloss-installable} Something that can be realised in the Nix store. See [installables](./command-ref/new-cli/nix.md#installables) for [`nix` commands](./command-ref/new-cli/nix.md) (experimental) for details. - [instantiate](#gloss-instantiate){#gloss-instantiate}, instantiation Translate a [derivation expression](#gloss-derivation-expression) into a [store derivation](#gloss-store-derivation). See [`nix-instantiate`](./command-ref/nix-instantiate.md), which produces a store derivation from a Nix expression that evaluates to a derivation. - \[Nix Archive (NAR)\]{#gloss-nar} A _N_ix _AR_chive. This is a serialisation of a path in the Nix store. It can contain regular files, directories and symbolic links. NARs are generated and unpacked using `nix-store --dump` and `nix-store --restore`. See [Nix Archive](store/file-system-object/content-address.html#serial-nix-archive) for details. - [Nix database](#gloss-nix-database){#gloss-nix-database} An SQlite database to track [reference](#gloss-reference)s between [store object](#gloss-store-object)s. This is an implementation detail of the [local store](@docroot@/store/types/local-store.md). Default location: `/nix/var/nix/db`. - \[Nix expression\]{#gloss-nix-expression} A syntactically valid use of the [Nix language](./language/index.md). > **Example** > > The contents of a `.nix` file form a Nix expression. Nix expressions specify [derivation expressions](#gloss-derivation-expression), which are [instantiated](#gloss-instantiate) into the Nix store as [store derivations](#gloss-store-derivation). These derivations can then be [realised](#gloss-realise) to produce [outputs](#gloss-output). > **Example** > > Building and deploying software using Nix entails writing Nix expressions to describe [packages](#package) and compositions thereof. - \[Nix instance\]{#gloss-nix-instance} 1. An installation of Nix, which includes the presence of a [store](#gloss-store), and the Nix package manager which operates on that store. A local Nix installation and a [remote builder](@docroot@/advanced-topics/distributed-builds.md) are two examples of Nix instances. 2. A running Nix process, such as the `nix` command. - [output](#gloss-output){#gloss-output} A [store object](#gloss-store-object) produced by a [store derivation](#gloss-store-derivation). See [the `outputs` argument to the `derivation` function](@docroot@/language/derivations.md#attr-outputs) for details. - \[output closure\]{#gloss-output-closure} The [closure](#gloss-closure) of an [output path](#gloss-output-path). It only contains what is \[reachable\] from the output. - [output path](#gloss-output-path){#gloss-output-path} The [store path](#gloss-store-path) to the [output](#gloss-output) of a [store derivation](#gloss-store-derivation). - [package](#package){#package} A software package; files that belong together for a particular purpose, and metadata. Nix represents files as [file system objects](#gloss-file-system-object), and how they belong together is encoded as [references](#gloss-reference) between [store objects](#gloss-store-object) that contain these file system objects. The [Nix language](./language/index.md) allows denoting packages in terms of [attribute sets](@docroot@/language/types.md#type-attrs) containing: - attributes that refer to the files of a package, typically in the form of [derivation outputs](#gloss-output), - attributes with metadata, such as information about how the package is supposed to be used. The exact shape of these attribute sets is up to convention. - \[profile\]{#gloss-profile} A symlink to the current _user environment_ of a user, e.g., `/nix/var/nix/profiles/default`. - \[purity\]{#gloss-purity} The assumption that equal Nix derivations when run always produce the same output. This cannot be guaranteed in general (e.g., a builder can rely on external inputs such as the network or the system time) but the Nix model assumes it. - \[reachable\]{#gloss-reachable} A store path `Q` is reachable from another store path `P` if `Q` is in the _closure_ of the _references_ relation. See [References](@docroot@/store/store-object.md#references) for details. - [realise](#gloss-realise){#gloss-realise}, realisation Ensure a [store path](#gloss-store-path) is [valid](#gloss-validity). This can be achieved by: - Fetching a pre-built [store object](#gloss-store-object) from a [substituter](#gloss-substituter) - [Building](@docroot@/store/building.md) the corresponding [store derivation](#gloss-store-derivation) - Delegating to a [remote machine](@docroot@/command-ref/conf-file.md#conf-builders) and retrieving the outputs See [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md) for a detailed description of the algorithm. See also [`nix-build`](./command-ref/nix-build.md) and [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental). - [reference](#gloss-reference){#gloss-reference} An edge from one [store object](#gloss-store-object) to another. See [References](@docroot@/store/store-object.md#references) for details. See [References](@docroot@/store/store-object.md#references) for details. - \[referrer\]{#gloss-referrer} A reversed edge from one [store object](#gloss-store-object) to another. - \[requisite\]{#gloss-requisite} A store object \[reachable\] by a path (chain of references) from a given [store object](#gloss-store-object). The [closure](#gloss-closure) is the set of requisites. See [References](@docroot@/store/store-object.md#references) for details. - [store](#gloss-store){#gloss-store} A collection of [store objects](#gloss-store-object), with operations to manipulate that collection. See [Nix Store](./store/index.md) for details. There are many types of stores, see [Store Types](./store/types/index.md) for details. - [store derivation](#gloss-store-derivation){#gloss-store-derivation} A [derivation](#gloss-derivation) represented as a [store object](#gloss-store-object). See [Store Derivation](@docroot@/store/derivation/index.md#store-derivation) for details. - [store object](#gloss-store-object){#gloss-store-object} Part of the contents of a [store](#gloss-store). A store object consists of a [file system object](#gloss-file-system-object), [references](#gloss-reference) to other store objects, and other metadata. It can be referred to by a [store path](#gloss-store-path). See [Store Object](@docroot@/store/store-object.md) for details. - [store path](#gloss-store-path){#gloss-store-path} The location of a [store object](#gloss-store-object) in the file system, i.e., an immediate child of the Nix store directory. > **Example** > > `/nix/store/jf6gn2dzna4nmsfbdxsd7kwhsk6gnnlr-git-2.38.1` See [Store Path](@docroot@/store/store-path.md) for details. - \[string interpolation\]{#gloss-string-interpolation} Expanding expressions enclosed in `${ }` within a [string](./language/types.md#type-string), [path](./language/types.md#type-path), or [attribute name](./language/types.md#type-attrs). See [String interpolation](./language/string-interpolation.md) for details. - [SRI](#gloss-sri){#gloss-sri} [Subresource Integrity](https://www.w3.org/TR/SRI/) (SRI) is a [W3C specification](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) for integrity metadata. Nix uses the SRI hash format (`-`) to specify content hashes in a way that is self-describing, since the hash algorithm is part of the format. - \[substitute\]{#gloss-substitute} A substitute is a command invocation stored in the [Nix database](#gloss-nix-database) that describes how to build a store object, bypassing the normal build mechanism (i.e., derivations). Typically, the substitute builds the store object by downloading a pre-built version of the store object from some server. - [substituter](#gloss-substituter){#gloss-substituter} An additional [store](#gloss-store){#gloss-store} from which Nix can obtain store objects instead of building them. Often the substituter is a [binary cache](#gloss-binary-cache), but any store can serve as substituter. See the [`substituters` configuration option](./command-ref/conf-file.md#conf-substituters) for details. - \[user environment\]{#gloss-user-env} An automatically generated store object that consists of a set of symlinks to “active” applications, i.e., other store paths. These are generated automatically by [`nix-env`](./command-ref/nix-env.md). See _profiles_. - [validity](#gloss-validity){#gloss-validity} A store path is valid if all [store object](#gloss-store-object)s in its [closure](#gloss-closure) can be read from the [store](#gloss-store). For a [local store](@docroot@/store/types/local-store.md), this means: - The store path leads to an existing [store object](#gloss-store-object) in that [store](#gloss-store). - The store path is listed in the [Nix database](#gloss-nix-database) as being valid. - All paths in the store path's [closure](#gloss-closure) are valid. ## [Installation](https://docharvest.github.io/docs/nix/installation/) Contents nix Installation Nix Installation This section describes how to install and configure Nix for first-time use. The current recommended option on Linux and MacOS is [multi-user](#multi-user). ## Multi-user This installation offers better sharing, improved isolation, and more security over a single user installation. This option requires either: - Linux running systemd, with SELinux disabled - MacOS > **Updating to macOS 15 Sequoia** > > If you recently updated to macOS 15 Sequoia and are getting > > ``` > error: the user '_nixbld1' in the group 'nixbld' does not exist > ``` > > when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling. ``` $ curl -L https://nixos.org/nix/install | sh -s -- --daemon ``` ## Single-user > Single-user is not supported on Mac. > `warning: installing Nix as root is not supported by this script!` This installation has less requirements than the multi-user install, however it cannot offer equivalent sharing, isolation, or security. This option is suitable for systems without systemd. ``` $ curl -L https://nixos.org/nix/install | sh -s -- --no-daemon ``` ## Distributions The Nix community maintains installers for several distributions. They can be found in the [`nix-community/nix-installers`](https://github.com/nix-community/nix-installers) repository. ## [Building Nix from Source](https://docharvest.github.io/docs/nix/installation/building-source/) Contents nix Building Nix from Source Nix Building Nix from Source Nix is built with [Meson](https://mesonbuild.com/). It is broken up into multiple Meson packages, which are optionally combined in a single project using Meson's [subprojects](https://mesonbuild.com/Subprojects.html) feature. There are no mandatory extra steps to the building process: generic Meson installation instructions like [this](https://mesonbuild.com/Quick-guide.html#using-meson-as-a-distro-packager) should work. ``` git clone https://github.com/NixOS/nix.git cd nix meson setup build cd build ninja (sudo) ninja install ``` The installation path can be specified by passing `-Dprefix=prefix` to `meson setup build`. The default installation directory is `/usr/local`. You can change this to any location you like. You must have write permission to the _prefix_ path. Nix keeps its _store_ (the place where packages are stored) in `/nix/store` by default. This can be changed using `-Dlibstore:store-dir=path`. > **Warning** > > It is best _not_ to change the Nix store from its default, since doing so makes it impossible to use pre-built binaries from the standard Nixpkgs channels — that is, all packages will need to be built from source. Nix keeps state (such as its database and log files) in `/nix/var` by default. This can be changed using `-Dlocalstatedir=path`. ## [Environment Variables](https://docharvest.github.io/docs/nix/installation/env-variables/) Contents nix Environment Variables Nix Environment Variables To use Nix, some environment variables should be set. In particular, `PATH` should contain the directories `prefix/bin` and `~/.nix-profile/bin`. The first directory contains the Nix tools themselves, while `~/.nix-profile` is a symbolic link to the current _user environment_ (an automatically generated package consisting of symlinks to installed packages). The simplest way to set the required environment variables is to include the file `prefix/etc/profile.d/nix.sh` in your `~/.profile` (or similar), like this: ``` source prefix/etc/profile.d/nix.sh ``` # `NIX_SSL_CERT_FILE` If you need to specify a custom certificate bundle to account for an HTTPS-intercepting man in the middle proxy, you must specify the path to the certificate bundle in the environment variable `NIX_SSL_CERT_FILE`. If you don't specify a `NIX_SSL_CERT_FILE` manually, Nix will install and use its own certificate bundle. Set the environment variable and install Nix ``` $ export NIX_SSL_CERT_FILE=/etc/ssl/my-certificate-bundle.crt $ curl -L https://nixos.org/nix/install | sh ``` In the shell profile and rc files (for example, `/etc/bashrc`, `/etc/zshrc`), add the following line: ``` export NIX_SSL_CERT_FILE=/etc/ssl/my-certificate-bundle.crt ``` > **Note** > > You must not add the export and then do the install, as the Nix installer will detect the presence of Nix configuration, and abort. If you use the Nix daemon, you should also add the following to `/etc/nix/nix.conf`: ``` ssl-cert-file = /etc/ssl/my-certificate-bundle.crt ``` ## Proxy Environment Variables The Nix installer has special handling for these proxy-related environment variables: `http_proxy`, `https_proxy`, `ftp_proxy`, `all_proxy`, `no_proxy`, `HTTP_PROXY`, `HTTPS_PROXY`, `FTP_PROXY`, `ALL_PROXY`, `NO_PROXY`. If any of these variables are set when running the Nix installer, then the installer will create an override file at `/etc/systemd/system/nix-daemon.service.d/override.conf` so `nix-daemon` will use them. ## [Installing a Binary Distribution](https://docharvest.github.io/docs/nix/installation/installing-binary/) Contents nix Installing a Binary Distribution Nix Installing a Binary Distribution > **Updating to macOS 15 Sequoia** > > If you recently updated to macOS 15 Sequoia and are getting > > ``` > error: the user '_nixbld1' in the group 'nixbld' does not exist > ``` > > when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling. To install the latest version Nix, run the following command: ``` $ curl -L https://nixos.org/nix/install | sh ``` This performs the default type of installation for your platform: - [Multi-user](#multi-user-installation): - Linux with systemd and without SELinux - macOS - [Single-user](#single-user-installation): - Linux without systemd - Linux with SELinux We recommend the multi-user installation if it supports your platform and you can authenticate with `sudo`. The installer can be configured with various command line arguments and environment variables. To show available command line flags: ``` $ curl -L https://nixos.org/nix/install | sh -s -- --help ``` To check what it does and how it can be customised further, [download and edit the second-stage installation script](#installing-from-a-binary-tarball). # Installing a pinned Nix version from a URL Version-specific installation URLs for all Nix versions since 1.11.16 can be found at [releases.nixos.org](https://releases.nixos.org/?prefix=nix/). The directory for each version contains the corresponding SHA-256 hash. All installation scripts are invoked the same way: ``` $ export VERSION=2.19.2 $ curl -L https://releases.nixos.org/nix/nix-$VERSION/install | sh ``` # Multi User Installation The multi-user Nix installation creates system users and a system service for the Nix daemon. Supported systems: - Linux running systemd, with SELinux disabled - macOS To explicitly instruct the installer to perform a multi-user installation on your system: ``` $ bash <(curl -L https://nixos.org/nix/install) --daemon ``` You can run this under your usual user account or `root`. The script will invoke `sudo` as needed. # Single User Installation To explicitly select a single-user installation on your system: ``` $ bash <(curl -L https://nixos.org/nix/install) --no-daemon ``` In a single-user installation, `/nix` is owned by the invoking user. The script will invoke `sudo` to create `/nix` if it doesn’t already exist. If you don’t have `sudo`, manually create `/nix` as `root`: ``` $ su root # mkdir /nix # chown alice /nix ``` # Installing from a binary tarball You can also download a binary tarball that contains Nix and all its dependencies: - Choose a [version](https://releases.nixos.org/?prefix=nix/) and [system type](../development/building.md#platforms) - Download and unpack the tarball - Run the installer > **Example** > > ``` > $ pushd $(mktemp -d) > $ export VERSION=2.19.2 > $ export SYSTEM=x86_64-linux > $ curl -LO https://releases.nixos.org/nix/nix-$VERSION/nix-$VERSION-$SYSTEM.tar.xz > $ tar xfj nix-$VERSION-$SYSTEM.tar.xz > $ cd nix-$VERSION-$SYSTEM > $ ./install > $ popd > ``` The installer can be customised with the environment variables declared in the file named `install-multi-user`. ## Native packages for Linux distributions The Nix community maintains installers for some Linux distributions in their native packaging format([https://nix-community.github.io/nix-installers/](https://nix-community.github.io/nix-installers/)). # macOS Installation \[\]{#sect-macos-installation-change-store-prefix}\[\]{#sect-macos-installation-encrypted-volume}\[\]{#sect-macos-installation-symlink}\[\]{#sect-macos-installation-recommended-notes} We believe we have ironed out how to cleanly support the read-only root file system on modern macOS. New installs will do this automatically. This section previously detailed the situation, options, and trade-offs, but it now only outlines what the installer does. You don't need to know this to run the installer, but it may help if you run into trouble: - create a new APFS volume for your Nix store - update `/etc/synthetic.conf` to direct macOS to create a "synthetic" empty root directory to mount your volume - specify mount options for the volume in `/etc/fstab` - `rw`: read-write - `noauto`: prevent the system from auto-mounting the volume (so the LaunchDaemon mentioned below can control mounting it, and to avoid masking problems with that mounting service). - `nobrowse`: prevent the Nix Store volume from showing up on your desktop; also keeps Spotlight from spending resources to index this volume - if you have FileVault enabled - generate an encryption password - put it in your system Keychain - use it to encrypt the volume - create a system LaunchDaemon to mount this volume early enough in the boot process to avoid problems loading or restoring any programs that need access to your Nix store ## [Using Nix within Docker](https://docharvest.github.io/docs/nix/installation/installing-docker/) Contents nix Using Nix within Docker Nix Using Nix within Docker To run the latest stable release of Nix with Docker run the following command: ``` $ docker run -ti docker.io/nixos/nix Unable to find image 'docker.io/nixos/nix:latest' locally latest: Pulling from docker.io/nixos/nix 5843afab3874: Pull complete b52bf13f109c: Pull complete 1e2415612aa3: Pull complete Digest: sha256:27f6e7f60227e959ee7ece361f75d4844a40e1cc6878b6868fe30140420031ff Status: Downloaded newer image for docker.io/nixos/nix:latest 35ca4ada6e96:/# nix --version nix (Nix) 2.3.12 35ca4ada6e96:/# exit ``` > If you want the latest pre-release you can use ghcr.io/nixos/nix and view them at [https://github.com/nixos/nix/pkgs/container/nix](https://github.com/nixos/nix/pkgs/container/nix) # What is included in Nix's Docker image? The official Docker image is created using `pkgs.dockerTools.buildLayeredImage` (and not with `Dockerfile` as it is usual with Docker images). You can still base your custom Docker image on it as you would do with any other Docker image. The Docker image is also not based on any other image and includes minimal set of runtime dependencies that are required to use Nix: - pkgs.nix - pkgs.bashInteractive - pkgs.coreutils-full - pkgs.gnutar - pkgs.gzip - pkgs.gnugrep - pkgs.which - pkgs.curl - pkgs.less - pkgs.wget - pkgs.man - pkgs.cacert.out - pkgs.findutils # Docker image with the latest development version of Nix To get the latest image that was built by [Hydra](https://hydra.nixos.org) run the following command: ``` $ curl -L https://hydra.nixos.org/job/nix/master/dockerImage.x86_64-linux/latest/download/1 | docker load $ docker run -ti nix:2.5pre20211105 ``` You can also build a Docker image from source yourself: ``` $ nix build ./\#hydraJobs.dockerImage.x86_64-linux $ docker load -i ./result/image.tar.gz $ docker run -ti nix:2.5pre20211105 ``` # Docker image with non-root Nix If you would like to run Nix in a container under a user other than `root`, you can build an image with a non-root single-user installation of Nix by specifying the `uid`, `gid`, `uname`, and `gname` arguments to `docker.nix`: ``` $ nix build --file docker.nix \ --arg uid 1000 \ --arg gid 1000 \ --argstr uname user \ --argstr gname user \ --argstr name nix-user \ --out-link nix-user.tar.gz $ docker load -i nix-user.tar.gz $ docker run -ti nix-user ``` ## [Installing Nix from Source](https://docharvest.github.io/docs/nix/installation/installing-source/) Contents nix Installing Nix from Source Nix Installing Nix from Source If no binary package is available or if you want to hack on Nix, you can build Nix from its Git repository. ## [Multi-User Mode](https://docharvest.github.io/docs/nix/installation/multi-user/) Contents nix Multi-User Mode Nix Multi-User Mode To allow a Nix store to be shared safely among multiple users, it is important that users are not able to run builders that modify the Nix store or database in arbitrary ways, or that interfere with builds started by other users. If they could do so, they could install a Trojan horse in some package and compromise the accounts of other users. To prevent this, the Nix store and database are owned by some privileged user (usually `root`) and builders are executed under special user accounts (usually named `nixbld1`, `nixbld2`, etc.). When a unprivileged user runs a Nix command, actions that operate on the Nix store (such as builds) are forwarded to a _Nix daemon_ running under the owner of the Nix store/database that performs the operation. > **Note** > > Multi-user mode has one important limitation: only root and a set of trusted users specified in `nix.conf` can specify arbitrary binary caches. So while unprivileged users may install packages from arbitrary Nix expressions, they may not get pre-built binaries. ## Setting up the build users The _build users_ are the special UIDs under which builds are performed. They should all be members of the _build users group_ `nixbld`. This group should have no other members. The build users should not be members of any other group. On Linux, you can create the group and users as follows: ``` $ groupadd -r nixbld $ for n in $(seq 1 10); do useradd -c "Nix build user $n" \ -d /var/empty -g nixbld -G nixbld -M -N -r -s "$(which nologin)" \ nixbld$n; done ``` This creates 10 build users. There can never be more concurrent builds than the number of build users, so you may want to increase this if you expect to do many builds at the same time. ## Running the daemon The [Nix daemon](../command-ref/nix-daemon.md) should be started as follows (as `root`): ``` $ nix-daemon ``` You’ll want to put that line somewhere in your system’s boot scripts. To let unprivileged users use the daemon, they should set the [`NIX_REMOTE` environment variable](../command-ref/env-common.md) to `daemon`. So you should put a line like ``` export NIX_REMOTE=daemon ``` into the users’ login scripts. ## Restricting access To limit which users can perform Nix operations, you can use the permissions on the directory `/nix/var/nix/daemon-socket`. For instance, if you want to restrict the use of Nix to the members of a group called `nix-users`, do ``` $ chgrp nix-users /nix/var/nix/daemon-socket $ chmod ug=rwx,o= /nix/var/nix/daemon-socket ``` This way, users who are not in the `nix-users` group cannot connect to the Unix domain socket `/nix/var/nix/daemon-socket/socket`, so they cannot perform Nix operations. ## [Nix Security](https://docharvest.github.io/docs/nix/installation/nix-security/) Contents nix Nix Security Nix Nix Security Nix has two basic security models. First, it can be used in “single-user mode”, which is similar to what most other package management tools do: there is a single user (typically root) who performs all package management operations. All other users can then use the installed packages, but they cannot perform package management operations themselves. Alternatively, you can configure Nix in “multi-user mode”. In this model, all users can perform package management operations — for instance, every user can install software without requiring root privileges. Nix ensures that this is secure. For instance, it’s not possible for one user to overwrite a package used by another user with a Trojan horse. ## [Obtaining the Source](https://docharvest.github.io/docs/nix/installation/obtaining-source/) Contents nix Obtaining the Source Nix Obtaining the Source The most recent sources of Nix can be obtained from its [Git repository](https://github.com/NixOS/nix). For example, the following command will check out the latest revision into a directory called `nix`: ``` $ git clone https://github.com/NixOS/nix ``` Likewise, specific releases can be obtained from the [tags](https://github.com/NixOS/nix/tags) of the repository. ## [Prerequisites Source](https://docharvest.github.io/docs/nix/installation/prerequisites-source/) Contents nix Prerequisites Source Nix Prerequisites Source This list and lower version bounds are maintained on best-effort basis. When in doubt, check the `meson.build` files. - Meson build system ([https://github.com/mesonbuild/meson](https://github.com/mesonbuild/meson)). - Ninja ([https://ninja-build.org/](https://ninja-build.org/)). - A version of GCC or Clang that supports C++23 (anything newer than Clang 19 or GCC 14 is likely to work). - `pkg-config` to locate dependencies. If your distribution does not provide it, you can get it from [http://www.freedesktop.org/wiki/Software/pkg-config](http://www.freedesktop.org/wiki/Software/pkg-config). - The OpenSSL library to calculate cryptographic hashes. If your distribution does not provide it, you can get it from [https://www.openssl.org](https://www.openssl.org). - The `libbrotlienc` and `libbrotlidec` libraries to provide implementation of the Brotli compression algorithm. They are available for download from the official repository [https://github.com/google/brotli](https://github.com/google/brotli). - cURL library. If your distribution does not provide it, you can get it from [https://curl.haxx.se/](https://curl.haxx.se/). - The SQLite embedded database library, version 3.6.19 or higher. If your distribution does not provide it, please install it from [http://www.sqlite.org/](http://www.sqlite.org/). - The [Boehm garbage collector (`bdw-gc`)](http://www.hboehm.info/gc/) to reduce the evaluator’s memory consumption (optional). To enable it, install `pkgconfig` and the Boehm garbage collector, and pass the option `-Dlibexpr:gc=enabled` to `meson setup`. - The `boost` library of version 1.87.0 or higher. It can be obtained from the official web site [https://www.boost.org/](https://www.boost.org/). - The `editline` library of version 1.14.0 or higher. It can be obtained from the its repository [https://github.com/troglobit/editline](https://github.com/troglobit/editline). - The `libsodium` library for verifying cryptographic signatures of contents fetched from binary caches. It can be obtained from the official web site [https://libsodium.org](https://libsodium.org). - Recent versions of Bison and Flex to build the parser. (This is because Nix needs C++ template support in Bison and reentrancy support in Flex.) - The `libseccomp` is used to provide syscall filtering on Linux. This is an optional dependency and can be disabled passing a `-Dlibstore:seccomp-sandboxing=disabled` option to the `meson setup` command (Not recommended unless your system doesn't support `libseccomp`). To get the library, visit [https://github.com/seccomp/libseccomp](https://github.com/seccomp/libseccomp). - On 64-bit x86 machines only, `libcpuid` library is used to determine which microarchitecture levels are supported (e.g., as whether to have `x86_64-v2-linux` among additional system types). The library is available from its homepage [http://libcpuid.sourceforge.net](http://libcpuid.sourceforge.net). This is an optional dependency and can be disabled by providing a `-Dlibutil:cpuid=disabled` option to `meson setup` script. - Unless `meson setup build -Dunit-tests=false` is specified, GoogleTest (GTest) and RapidCheck are required, which are available at [https://google.github.io/googletest/](https://google.github.io/googletest/) and [https://github.com/emil-e/rapidcheck](https://github.com/emil-e/rapidcheck) respectively. ## [Single-User Mode](https://docharvest.github.io/docs/nix/installation/single-user/) Contents nix Single-User Mode Nix Single-User Mode In single-user mode, all Nix operations that access the database in `prefix/var/nix/db` or modify the Nix store in `prefix/store` must be performed under the user ID that owns those directories. This is typically root. (If you install from RPM packages, that’s in fact the default ownership.) However, on single-user machines, it is often convenient to `chown` those directories to your normal user account so that you don’t have to `su` to root all the time. ## [Supported Platforms](https://docharvest.github.io/docs/nix/installation/supported-platforms/) Contents nix Supported Platforms Nix Supported Platforms Nix is currently supported on the following platforms: - Linux (i686, x86\_64, aarch64). - macOS (x86\_64, aarch64). ## [Uninstalling Nix](https://docharvest.github.io/docs/nix/installation/uninstall/) Contents nix Uninstalling Nix Nix Uninstalling Nix ## Multi User Removing a [multi-user installation](./installing-binary.md#multi-user-installation) depends on the operating system. ### Linux If you are on Linux with systemd: 1. Remove the Nix daemon service: ``` sudo systemctl stop nix-daemon.service sudo systemctl disable nix-daemon.socket nix-daemon.service sudo systemctl daemon-reload ``` 2. Remove files created by Nix: ``` sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~/.local/share/nix ~/.local/state/nix ~/.cache/nix ~/.nix-defexpr ~/.nix-profile ~/.nix-channels ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix ``` 3. Remove build users and their group: ``` for i in $(seq 1 32); do sudo userdel nixbld$i done sudo groupdel nixbld ``` 4. There may also be references to Nix in - `/etc/bash.bashrc` - `/etc/bashrc` - `/etc/profile` - `/etc/zsh/zshrc` - `/etc/zshrc` which you may remove. ### FreeBSD 1. Stop and remove the Nix daemon service: ``` sudo service nix-daemon stop sudo rm -f /usr/local/etc/rc.d/nix-daemon sudo sysrc -x nix_daemon_enable ``` 2. Remove files created by Nix: ``` sudo rm -rf /etc/nix /usr/local/etc/profile.d/nix.sh /nix ~/.local/share/nix ~/.local/state/nix ~/.cache/nix ~/.nix-defexpr ~/.nix-profile ~/.nix-channels ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix ``` 3. Remove build users and their group: ``` for i in $(seq 1 32); do sudo pw userdel nixbld$i done sudo pw groupdel nixbld ``` 4. There may also be references to Nix in: - `/usr/local/etc/bashrc` - `/usr/local/etc/zshrc` - Shell configuration files in users' home directories which you may remove. ### macOS > **Updating to macOS 15 Sequoia** > > If you recently updated to macOS 15 Sequoia and are getting > > ``` > error: the user '_nixbld1' in the group 'nixbld' does not exist > ``` > > when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling. 1. If system-wide shell initialisation files haven't been altered since installing Nix, use the backups made by the installer: ``` sudo mv /etc/zshrc.backup-before-nix /etc/zshrc sudo mv /etc/bashrc.backup-before-nix /etc/bashrc sudo mv /etc/bash.bashrc.backup-before-nix /etc/bash.bashrc ``` Otherwise, edit `/etc/zshrc`, `/etc/bashrc`, and `/etc/bash.bashrc` to remove the lines sourcing `nix-daemon.sh`, which should look like this: ``` # Nix if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' fi # End Nix ``` 2. Stop and remove the Nix daemon services: ``` sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist sudo launchctl unload /Library/LaunchDaemons/org.nixos.darwin-store.plist sudo rm /Library/LaunchDaemons/org.nixos.darwin-store.plist ``` This stops the Nix daemon and prevents it from being started next time you boot the system. 3. Remove the `nixbld` group and the `_nixbuildN` users: ``` sudo dscl . -delete /Groups/nixbld for u in $(sudo dscl . -list /Users | grep _nixbld); do sudo dscl . -delete /Users/$u; done ``` This will remove all the build users that no longer serve a purpose. 4. Edit fstab using `sudo vifs` to remove the line mounting the Nix Store volume on `/nix`, which looks like ``` UUID= /nix apfs rw,noauto,nobrowse,suid,owners ``` or ``` LABEL=Nix\040Store /nix apfs rw,nobrowse ``` by setting the cursor on the respective line using the arrow keys, and pressing `dd`, and then `:wq` to save the file. This will prevent automatic mounting of the Nix Store volume. 5. Edit `/etc/synthetic.conf` to remove the `nix` line. If this is the only line in the file you can remove it entirely: ``` if [ -f /etc/synthetic.conf ]; then if [ "$(cat /etc/synthetic.conf)" = "nix" ]; then sudo rm /etc/synthetic.conf else sudo vi /etc/synthetic.conf fi fi ``` This will prevent the creation of the empty `/nix` directory. 6. Remove the files Nix added to your system, except for the store: ``` sudo rm -rf /etc/nix /var/root/.nix-profile /var/root/.nix-defexpr /var/root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels ~/.local/share/nix ~/.local/state/nix ~/.cache/nix ``` 7. Remove the Nix Store volume: ``` sudo diskutil apfs deleteVolume /nix ``` This will remove the Nix Store volume and everything that was added to the store. If the output indicates that the command couldn't remove the volume, you should make sure you don't have an _unmounted_ Nix Store volume. Look for a "Nix Store" volume in the output of the following command: ``` diskutil list ``` If you _do_ find a "Nix Store" volume, delete it by running `diskutil apfs deleteVolume` with the store volume's `diskXsY` identifier. If you get an error that the volume is in use by the kernel, reboot and immediately delete the volume before starting any other process. > **Note** > > After you complete the steps here, you will still have an empty `/nix` directory. This is an expected sign of a successful uninstall. The empty `/nix` directory will disappear the next time you reboot. > > You do not have to reboot to finish uninstalling Nix. The uninstall is complete. macOS (Catalina+) directly controls root directories, and its read-only root will prevent you from manually deleting the empty `/nix` mountpoint. ## Single User To remove a [single-user installation](./installing-binary.md#single-user-installation) of Nix, run: ``` rm -rf /nix ~/.nix-channels ~/.nix-defexpr ~/.nix-profile ~/.local/share/nix ~/.local/state/nix ~/.cache/nix ``` You might also want to manually remove references to Nix from your `~/.profile`. ## [Upgrading Nix](https://docharvest.github.io/docs/nix/installation/upgrading/) Contents nix Upgrading Nix Nix Upgrading Nix > **Note** > > These upgrade instructions apply where Nix was installed following the [installation instructions in this manual](./index.md). Check which Nix version will be installed, for example from one of the [release channels](http://channels.nixos.org/) such as `nixpkgs-unstable`: ``` $ nix-shell -p nix -I nixpkgs=channel:nixpkgs-unstable --run "nix --version" nix (Nix) 2.18.1 ``` > **Warning** > > Writing to the [local store](@docroot@/store/types/local-store.md) with a newer version of Nix, for example by building derivations with [`nix-build`](@docroot@/command-ref/nix-build.md) or [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md), may change the database schema! Reverting to an older version of Nix may therefore require purging the store database before it can be used. ## Linux multi-user ``` $ sudo su # nix-env --install --file '' --attr nix cacert -I nixpkgs=channel:nixpkgs-unstable # systemctl daemon-reload # systemctl restart nix-daemon ``` ## macOS multi-user ``` $ sudo nix-env --install --file '' --attr nix cacert -I nixpkgs=channel:nixpkgs-unstable $ sudo launchctl remove org.nixos.nix-daemon $ sudo launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist ``` ## Single-user all platforms ``` $ nix-env --install --file '' --attr nix cacert -I nixpkgs=channel:nixpkgs-unstable ``` ## [Introduction](https://docharvest.github.io/docs/nix/introduction/) Contents nix Introduction Nix Introduction Nix is a _purely functional package manager_. This means that it treats packages like values in a purely functional programming language — packages are built by functions that don’t have side-effects, and they never change after they have been built. Nix stores packages in the _Nix store_, usually the directory `/nix/store`, where each package has its own unique subdirectory such as ``` /nix/store/q06x3jll2yfzckz2bzqak089p43ixkkq-firefox-33.1/ ``` where `q06x3jll2yfz…` is a unique identifier for the package that captures all its dependencies (it’s a cryptographic hash of the package’s build dependency graph). This enables many powerful features. ## Multiple versions You can have multiple versions or variants of a package installed at the same time. This is especially important when different applications have dependencies on different versions of the same package — it prevents the “DLL hell”. Because of the hashing scheme, different versions of a package end up in different paths in the Nix store, so they don’t interfere with each other. An important consequence is that operations like upgrading or uninstalling an application cannot break other applications, since these operations never “destructively” update or delete files that are used by other packages. ## Complete dependencies Nix helps you make sure that package dependency specifications are complete. In general, when you’re making a package for a package management system like RPM, you have to specify for each package what its dependencies are, but there are no guarantees that this specification is complete. If you forget a dependency, then the package will build and work correctly on _your_ machine if you have the dependency installed, but not on the end user's machine if it's not there. Since Nix on the other hand doesn’t install packages in “global” locations like `/usr/bin` but in package-specific directories, the risk of incomplete dependencies is greatly reduced. This is because tools such as compilers don’t search in per-packages directories such as `/nix/store/5lbfaxb722zp…-openssl-0.9.8d/include`, so if a package builds correctly on your system, this is because you specified the dependency explicitly. This takes care of the build-time dependencies. Once a package is built, runtime dependencies are found by scanning binaries for the [hash parts](@docroot@/store/store-path.md#digest) of Nix store paths (such as `r8vvq9kq…`). This sounds risky, but it works extremely well. ## Multi-user support Nix has multi-user support. This means that non-privileged users can securely install software. Each user can have a different _profile_, a set of packages in the Nix store that appear in the user’s `PATH`. If a user installs a package that another user has already installed previously, the package won’t be built or downloaded a second time. At the same time, it is not possible for one user to inject a Trojan horse into a package that might be used by another user. ## Atomic upgrades and rollbacks Since package management operations never overwrite packages in the Nix store but just add new versions in different paths, they are _atomic_. So during a package upgrade, there is no time window in which the package has some files from the old version and some files from the new version — which would be bad because a program might well crash if it’s started during that period. And since packages aren’t overwritten, the old versions are still there after an upgrade. This means that you can _roll back_ to the old version: ``` $ nix-env --upgrade --attr nixpkgs.some-package $ nix-env --rollback ``` ## Garbage collection When you uninstall a package like this… ``` $ nix-env --uninstall firefox ``` the package isn’t deleted from the system right away (after all, you might want to do a rollback, or it might be in the profiles of other users). Instead, unused packages can be deleted safely by running the _garbage collector_: ``` $ nix-collect-garbage ``` This deletes all packages that aren’t in use by any user profile or by a currently running program. ## Functional package language Packages are built from _Nix expressions_, which is a simple functional language. A Nix expression describes everything that goes into a package build task (a “derivation”): other packages, sources, the build script, environment variables for the build script, etc. Nix tries very hard to ensure that Nix expressions are _deterministic_: building a Nix expression twice should yield the same result. Because it’s a functional language, it’s easy to support building variants of a package: turn the Nix expression into a function and call it any number of times with the appropriate arguments. Due to the hashing scheme, variants don’t conflict with each other in the Nix store. ## Transparent source/binary deployment Nix expressions generally describe how to build a package from source, so an installation action like ``` $ nix-env --install --attr nixpkgs.firefox ``` _could_ cause quite a bit of build activity, as not only Firefox but also all its dependencies (all the way up to the C library and the compiler) would have to be built, at least if they are not already in the Nix store. This is a _source deployment model_. For most users, building from source is not very pleasant as it takes far too long. However, Nix can automatically skip building from source and instead use a _binary cache_, a web server that provides pre-built binaries. For instance, when asked to build `/nix/store/b6gvzjyb2pg0…-firefox-33.1` from source, Nix would first check if the file `https://cache.nixos.org/b6gvzjyb2pg0….narinfo` exists, and if so, fetch the pre-built binary referenced from there; otherwise, it would fall back to building from source. ## Nix Packages collection We provide a large set of Nix expressions containing hundreds of existing Unix packages, the _Nix Packages collection_ (Nixpkgs). ## Managing build environments Nix is extremely useful for developers as it makes it easy to automatically set up the build environment for a package. Given a Nix expression that describes the dependencies of your package, the command `nix-shell` will build or download those dependencies if they’re not already in your Nix store, and then start a Bash shell in which all necessary environment variables (such as compiler search paths) are set. For example, the following command gets all dependencies of the Pan newsreader, as described by [its Nix expression](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/newsreaders/pan/default.nix): ``` $ nix-shell '' --attr pan ``` You’re then dropped into a shell where you can edit, build and test the package: ``` [nix-shell]$ unpackPhase [nix-shell]$ cd pan-* [nix-shell]$ configurePhase [nix-shell]$ buildPhase [nix-shell]$ ./pan/gui/pan ``` ## Portability Nix runs on Linux, macOS and FreeBSD. ## NixOS NixOS is a Linux distribution based on Nix. It uses Nix not just for package management but also to manage the system configuration (e.g., to build configuration files in `/etc`). This means, among other things, that it is easy to roll back the entire configuration of the system to an earlier state. Also, users can install software without root privileges. For more information and downloads, see the [NixOS homepage](https://nixos.org/). ## License Nix is released under the terms of the [GNU LGPLv2.1 or (at your option) any later version](http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html). ## [Nix Language](https://docharvest.github.io/docs/nix/language/) Contents nix Nix Language Nix Nix Language The Nix language is designed for conveniently creating and composing [derivations](@docroot@/glossary.md#gloss-derivation) – precise descriptions of how contents of existing files are used to derive new files. > **Tip** > > These pages are written as a reference. If you are learning Nix, nix.dev has a good [introduction to the Nix language](https://nix.dev/tutorials/nix-language). The language is: - _domain-specific_ The Nix language is purpose-built for working with text files. Its most characteristic features are: - [File system path primitives](@docroot@/language/types.md#type-path), for accessing source files - [Indented strings](@docroot@/language/string-literals.md) and [string interpolation](@docroot@/language/string-interpolation.md), for creating file contents - [Strings with contexts](@docroot@/language/string-context.md), for transparently linking files It comes with [built-in functions](@docroot@/language/builtins.md) to integrate with the [Nix store](@docroot@/store/index.md), which manages files and enables [realising](@docroot@/glossary.md#gloss-realise) derivations declared in the Nix language. - _declarative_ There is no notion of executing sequential steps. Dependencies between operations are established only through data. - _pure_ Values cannot change during computation. Functions always produce the same output if their input does not change. - _functional_ Functions are like any other value. Functions can be assigned to names, taken as arguments, or returned by functions. - _lazy_ Values are only computed when they are needed. - _dynamically typed_ Type errors are only detected when expressions are evaluated. # Overview This is an incomplete overview of language features, by example. Example Description _Basic values ([primitives](@docroot@/language/types.md#primitives))_ `"hello world"` A [string](@docroot@/language/types.md#type-string) ``` '' multi line string '' ``` A multi-line string. Strips common prefixed whitespace. Evaluates to `"multi\n line\n  string"`. `# Explanation` A [comment](@docroot@/language/syntax.md#comments). `"hello ${ { a = "world"; }.a }"` `"1 2 ${toString 3}"` `"${pkgs.bash}/bin/sh"` [String interpolation](@docroot@/language/string-interpolation.md) (expands to `"hello world"`, `"1 2 3"`, `"/nix/store/-bash-/bin/sh"`) `true`, `false` [Booleans](@docroot@/language/types.md#type-bool) `null` [Null](@docroot@/language/types.md#type-null) value `123` An [integer](@docroot@/language/types.md#type-int) `3.141` A [floating point number](@docroot@/language/types.md#type-float) `/etc` An absolute [path](@docroot@/language/types.md#type-path) `./foo.png` A [path](@docroot@/language/types.md#type-path) relative to the file containing this Nix expression `~/.config` A home [path](@docroot@/language/types.md#type-path). Evaluates to the `"/.config"`. `` A [lookup path](@docroot@/language/constructs/lookup-path.md) for Nix files. Value determined by [`$NIX_PATH` environment variable](../command-ref/env-common.md#env-NIX_PATH). _Compound values_ `{ x = 1; y = 2; }` An [attribute set](@docroot@/language/types.md#type-attrs) with attributes named `x` and `y` `{ foo.bar = 1; }` A nested set, equivalent to `{ foo = { bar = 1; }; }` `rec { x = "foo"; y = x + "bar"; }` A [recursive set](@docroot@/language/syntax.md#recursive-sets), equivalent to `{ x = "foo"; y = "foobar"; }`. `[ "foo" "bar" "baz" ]` `[ 1 2 3 ]` `[ (f 1) { a = 1; b = 2; } [ "c" ] ]` [Lists](@docroot@/language/types.md#type-list) with three elements. _Operators_ `"foo" + "bar"` String concatenation `1 + 2` Integer addition `"foo" == "f" + "oo"` Equality test (evaluates to `true`) `"foo" != "bar"` Inequality test (evaluates to `true`) `!true` Boolean negation `{ x = 1; y = 2; }.x` [Attribute selection](@docroot@/language/types.md#type-attrs) (evaluates to `1`) `{ x = 1; y = 2; }.z or 3` [Attribute selection](@docroot@/language/types.md#type-attrs) with default (evaluates to `3`) `{ x = 1; y = 2; } // { z = 3; }` Merge two sets (attributes in the right-hand set taking precedence) _Control structures_ `if 1 + 1 == 2 then "yes!" else "no!"` [Conditional expression](@docroot@/language/syntax.md#conditionals). `assert 1 + 1 == 2; "yes!"` [Assertion](@docroot@/language/syntax.md#assertions) check (evaluates to `"yes!"`). `let x = "foo"; y = "bar"; in x + y` Variable definition. See [`let`\-expressions](@docroot@/language/syntax.md#let-expressions). `with builtins; head [ 1 2 3 ]` Add all attributes from the given set to the scope (evaluates to `1`). See [`with`\-expressions](@docroot@/language/syntax.md#with-expressions) for details and shadowing caveats. `inherit pkgs src;` Adds the variables to the current scope (attribute set or `let` binding). Desugars to `pkgs = pkgs; src = src;`. See [Inheriting attributes](@docroot@/language/syntax.md#inheriting-attributes). `inherit (pkgs) lib stdenv;` Adds the attributes, from the attribute set in parentheses, to the current scope (attribute set or `let` binding). Desugars to `lib = pkgs.lib; stdenv = pkgs.stdenv;`. See [Inheriting attributes](@docroot@/language/syntax.md#inheriting-attributes). _[Functions](@docroot@/language/syntax.md#functions) (lambdas)_ `x: x + 1` A [function](@docroot@/language/syntax.md#functions) that expects an integer and returns it increased by 1. `x: y: x + y` Curried [function](@docroot@/language/syntax.md#functions), equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum. `(x: x + 1) 100` A [function](@docroot@/language/syntax.md#functions) call (evaluates to 101) `let inc = x: x + 1; in inc (inc (inc 100))` A [function](@docroot@/language/syntax.md#functions) bound to a variable and subsequently called by name (evaluates to 103) `{ x, y }: x + y` A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and concatenates them `{ x, y ? "bar" }: x + y` A [function](@docroot@/language/syntax.md#functions) that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y` `{ x, y, ... }: x + y` A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and ignores any other attributes `{ x, y } @ args: x + y` `args @ { x, y }: x + y` A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y`, and binds the whole set to `args` _Built-in functions_ `import ./foo.nix` Load and return Nix expression in given file. See [import](@docroot@/language/builtins.md#builtins-import). `map (x: x + x) [ 1 2 3 ]` Apply a function to every element of a list (evaluates to `[ 2 4 6 ]`). See [`map`](@docroot@/language/builtins.md#builtins-map). ## [Advanced Attributes](https://docharvest.github.io/docs/nix/language/advanced-attributes/) Contents nix Advanced Attributes Nix Advanced Attributes Derivations can declare some infrequently used optional attributes. ## Inputs - \[`exportReferencesGraph`\]{#adv-attr-exportReferencesGraph} This attribute allows builders access to the references graph of their inputs. The attribute is a list of inputs in the Nix store whose references graph the builder needs to know. The value of this attribute should be a list of pairs `[ name1 path1 name2 path2 ... ]`. The references graph of each _pathN_ will be stored in a text file _nameN_ in the temporary build directory. The text files have the format used by `nix-store --register-validity` (with the deriver fields left empty). For example, when the following derivation is built: ``` derivation { ... exportReferencesGraph = [ "libfoo-graph" libfoo ]; }; ``` the references graph of `libfoo` is placed in the file `libfoo-graph` in the temporary build directory. `exportReferencesGraph` is useful for builders that want to do something with the closure of a store path. Examples include the builders in NixOS that generate the initial ramdisk for booting Linux (a `cpio` archive containing the closure of the boot script) and the ISO-9660 image for the installation CD (which is populated with a Nix store containing the closure of a bootable NixOS configuration). - \[`passAsFile`\]{#adv-attr-passAsFile} A list of names of attributes that should be passed via files rather than environment variables. For example, if you have ``` passAsFile = ["big"]; big = "a very long string"; ``` then when the builder runs, the environment variable `bigPath` will contain the absolute path to a temporary file containing `a very long string`. That is, for any attribute _x_ listed in `passAsFile`, Nix will pass an environment variable `xPath` holding the path of the file containing the value of attribute _x_. This is useful when you need to pass large strings to a builder, since most operating systems impose a limit on the size of the environment (typically, a few hundred kilobyte). - \[`__structuredAttrs`\]{#adv-attr-structuredAttrs} If the special attribute `__structuredAttrs` is set to `true`, the other derivation attributes are serialised into a file in JSON format. This obviates the need for [`passAsFile`](#adv-attr-passAsFile) since JSON files have no size restrictions, unlike process environments. It also makes it possible to tweak derivation settings in a structured way; see [`outputChecks`](#adv-attr-outputChecks) for example. See the [corresponding section in the derivation page](@docroot@/store/derivation/index.md#structured-attrs) for further details. > **Warning** > > If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites), [`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize. will have no effect. ## Output checks See the [corresponding section in the derivation output page](@docroot@/store/derivation/outputs/index.md). - \[`allowedReferences`\]{#adv-attr-allowedReferences} The optional attribute `allowedReferences` specifies a list of legal references (dependencies) of the output of the builder. For example, ``` allowedReferences = []; ``` enforces that the output of a derivation cannot have any runtime dependencies on its inputs. To allow an output to have a runtime dependency on itself, use `"out"` as a list item. This is used in NixOS to check that generated files such as initial ramdisks for booting Linux don’t have accidental dependencies on other paths in the Nix store. - \[`allowedRequisites`\]{#adv-attr-allowedRequisites} This attribute is similar to `allowedReferences`, but it specifies the legal requisites of the whole closure, so all the dependencies recursively. For example, ``` allowedRequisites = [ foobar ]; ``` enforces that the output of a derivation cannot have any other runtime dependency than `foobar`, and in addition it enforces that `foobar` itself doesn't introduce any other dependency itself. - \[`disallowedReferences`\]{#adv-attr-disallowedReferences} The optional attribute `disallowedReferences` specifies a list of illegal references (dependencies) of the output of the builder. For example, ``` disallowedReferences = [ foo ]; ``` enforces that the output of a derivation cannot have a direct runtime dependencies on the derivation `foo`. - \[`disallowedRequisites`\]{#adv-attr-disallowedRequisites} This attribute is similar to `disallowedReferences`, but it specifies illegal requisites for the whole closure, so all the dependencies recursively. For example, ``` disallowedRequisites = [ foobar ]; ``` enforces that the output of a derivation cannot have any runtime dependency on `foobar` or any other derivation depending recursively on `foobar`. - \[`outputChecks`\]{#adv-attr-outputChecks} When using [structured attributes](#adv-attr-structuredAttrs), the `outputChecks` attribute allows defining checks per-output. In addition to [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites), [`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), the following attributes are available: - `maxSize` defines the maximum size of the resulting [store object](@docroot@/store/store-object.md). - `maxClosureSize` defines the maximum size of the output's closure. - `ignoreSelfRefs` controls whether self-references should be considered when checking for allowed references/requisites. Example: ``` __structuredAttrs = true; outputChecks.out = { # The closure of 'out' must not be larger than 256 MiB. maxClosureSize = 256 * 1024 * 1024; # It must not refer to the C compiler or to the 'dev' output. disallowedRequisites = [ stdenv.cc "dev" ]; }; outputChecks.dev = { # The 'dev' output must not be larger than 128 KiB. maxSize = 128 * 1024; }; ``` ## Other output modifications - \[`unsafeDiscardReferences`\]{#adv-attr-unsafeDiscardReferences} When using [structured attributes](#adv-attr-structuredAttrs), the attribute `unsafeDiscardReferences` is an attribute set with a boolean value for each output name. If set to `true`, it disables scanning the output for runtime dependencies. Example: ``` __structuredAttrs = true; unsafeDiscardReferences.out = true; ``` This is useful, for example, when generating self-contained filesystem images with their own embedded Nix store: hashes found inside such an image refer to the embedded store and not to the host's Nix store. ## Build scheduling - \[`preferLocalBuild`\]{#adv-attr-preferLocalBuild} If this attribute is set to `true` and [distributed building is enabled](@docroot@/command-ref/conf-file.md#conf-builders), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine. This is useful for derivations that are cheapest to build locally. - \[`allowSubstitutes`\]{#adv-attr-allowSubstitutes} If this attribute is set to `false`, then Nix will always build this derivation (locally or remotely); it will not try to substitute its outputs. This is useful for derivations that are cheaper to build than to substitute. This attribute can be ignored by setting [`always-allow-substitutes`](@docroot@/command-ref/conf-file.md#conf-always-allow-substitutes) to `true`. > **Note** > > If set to `false`, the [`builder`](./derivations.md#attr-builder) should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted. - \[`requiredSystemFeatures`\]{#adv-attr-requiredSystemFeatures} If a derivation has the `requiredSystemFeatures` attribute, then Nix will only build it on a machine that has the corresponding features set in its [`system-features` configuration](@docroot@/command-ref/conf-file.md#conf-system-features). For example, setting ``` requiredSystemFeatures = [ "kvm" ]; ``` ensures that the derivation can only be built on a machine with the `kvm` feature. # Impure builder configuration - \[`impureEnvVars`\]{#adv-attr-impureEnvVars} This attribute allows you to specify a list of environment variables that should be passed from the environment of the calling user to the builder. Usually, the environment is cleared completely when the builder is executed, but with this attribute you can allow specific environment variables to be passed unmodified. For example, `fetchurl` in Nixpkgs has the line ``` impureEnvVars = [ "http_proxy" "https_proxy" ... ]; ``` to make it use the proxy server configuration specified by the user in the environment variables `http_proxy` and friends. This attribute is only allowed in [fixed-output derivations](@docroot@/glossary.md#gloss-fixed-output-derivation), where impurities such as these are okay since (the hash of) the output is known in advance. It is ignored for all other derivations. > **Warning** > > `impureEnvVars` implementation takes environment variables from the current builder process. When a daemon is building its environmental variables are used. Without the daemon, the environmental variables come from the environment of the `nix-build`. If the [`configurable-impure-env` experimental feature](@docroot@/development/experimental-features.md#xp-feature-configurable-impure-env) is enabled, these environment variables can also be controlled through the [`impure-env`](@docroot@/command-ref/conf-file.md#conf-impure-env) configuration setting. ## Setting the derivation type As discussed in [Derivation Outputs and Types of Derivations](@docroot@/store/derivation/outputs/index.md), there are multiples kinds of derivations / kinds of derivation outputs. The choice of the following attributes determines which kind of derivation we are making. - [`__contentAddressed`](#adv-attr-__contentAddressed) - [`outputHash`](#adv-attr-outputHash) - [`outputHashAlgo`](#adv-attr-outputHashAlgo) - [`outputHashMode`](#adv-attr-outputHashMode) The three types of derivations are chosen based on the following combinations of these attributes. All other combinations are invalid. - [Input-addressing derivations](@docroot@/store/derivation/outputs/input-address.md) This is the default for `builtins.derivation`. Nix only currently supports one kind of input-addressing, so no other information is needed. `__contentAddressed = false;` may also be included, but is not needed, and will trigger the experimental feature check. - [Fixed-output derivations](@docroot@/glossary.md#gloss-fixed-output-derivation) All of [`outputHash`](#adv-attr-outputHash), [`outputHashAlgo`](#adv-attr-outputHashAlgo), and [`outputHashMode`](#adv-attr-outputHashMode). - [(Floating) content-addressing derivations](@docroot@/store/derivation/outputs/content-address.md) Both [`outputHashAlgo`](#adv-attr-outputHashAlgo) and [`outputHashMode`](#adv-attr-outputHashMode), `__contentAddressed = true;`, and _not_ `outputHash`. If an output hash was given, then the derivation output would be "fixed" not "floating". Here is more information on the `output*` attributes, and what values they may be set to: - [`outputHashMode`](#adv-attr-outputHashMode){#adv-attr-outputHashMode} This specifies how the files of a content-addressing derivation output are digested to produce a content address. This works in conjunction with [`outputHashAlgo`](#adv-attr-outputHashAlgo). Specifying one without the other is an error (unless \[`outputHash` is also specified and includes its own hash algorithm as described below). The `outputHashMode` attribute determines how the hash is computed. It must be one of the following values: - [`"flat"`](@docroot@/store/store-object/content-address.md#method-flat) This is the default. - [`"recursive"` or `"nar"`](@docroot@/store/store-object/content-address.md#method-nix-archive) > **Compatibility** > > `"recursive"` is the traditional way of indicating this, and is supported since 2005 (virtually the entire history of Nix). `"nar"` is more clear, and consistent with other parts of Nix (such as the CLI), however support for it is only added in Nix version 2.21. - [`"text"`](@docroot@/store/store-object/content-address.md#method-text) > **Warning** > > The use of this method for derivation outputs is part of the [`dynamic-derivations`](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) experimental feature. - [`"git"`](@docroot@/store/store-object/content-address.md#method-git) > **Warning** > > This method is part of the [`git-hashing`](@docroot@/development/experimental-features.md#xp-feature-git-hashing) experimental feature. See [content-addressing store objects](@docroot@/store/store-object/content-address.md) for more information about the process this flag controls. - [`outputHashAlgo`](#adv-attr-outputHashAlgo){#adv-attr-outputHashAlgo} This specifies the hash algorithm used to digest the [file system object](@docroot@/store/file-system-object.md) data of a content-addressing derivation output. This works in conjunction with [`outputHashMode`](#adv-attr-outputHashAlgo). Specifying one without the other is an error (unless `outputHash` is also specified and includes its own hash algorithm as described below). The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash. It can currently be `"blake3"`, `"sha1"`, `"sha256"`, `"sha512"`, or `null`. `outputHashAlgo` can only be `null` when `outputHash` follows the SRI format, because in that case the choice of hash algorithm is determined by `outputHash`. - [`outputHash`](#adv-attr-outputHash){#adv-attr-outputHash} This will specify the output hash of the single output of a [fixed-output derivation](@docroot@/glossary.md#gloss-fixed-output-derivation). The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](@docroot@/glossary.md#gloss-sri). The ["nix32" encoding](@docroot@/protocols/nix32.md) is Nix's variant of Base32 encoding. > **Note** > > The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings. The [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings. - [`__contentAddressed`](#adv-attr-__contentAddressed){#adv-attr-\_\_contentAddressed} > **Warning** > > This attribute is part of an [experimental feature](@docroot@/development/experimental-features.md). > > To use this attribute, you must enable the \[`ca-derivations`\]\[xp-feature-ca-derivations\] experimental feature. For example, in [nix.conf](../command-ref/conf-file.md) you could add: > > ``` > extra-experimental-features = ca-derivations > ``` This is a boolean with a default of `false`. It determines whether the derivation is floating content-addressing. ## [Builtins Prefix](https://docharvest.github.io/docs/nix/language/builtins-prefix/) ## [Builtins Suffix](https://docharvest.github.io/docs/nix/language/builtins-suffix/) Contents nix Builtins Suffix Nix Builtins Suffix ## [Language Constructs](https://docharvest.github.io/docs/nix/language/constructs/) Contents nix Language Constructs Nix Language Constructs ## [Lookup path](https://docharvest.github.io/docs/nix/language/constructs/lookup-path/) Contents nix Lookup path Nix Lookup path > **Syntax** > > _lookup-path_ = `<` _identifier_ \[ `/` _identifier_ \]... `>` A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/types.md#type-path) if the identifier matches a search path entry in [`builtins.nixPath`](@docroot@/language/builtins.md#builtins-nixPath). The algorithm for lookup path resolution is described in the documentation on [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile). > **Example** > > ``` > > ``` > > ``` > /nix/var/nix/profiles/per-user/root/channels/nixpkgs > ``` > **Example** > > ``` > > ``` > > ``` > /nix/var/nix/profiles/per-user/root/channels/nixpkgs/nixos > ``` ## [Derivations](https://docharvest.github.io/docs/nix/language/derivations/) Contents nix Derivations Nix Derivations The most important built-in function is `derivation`, which is used to describe a single store-layer [store derivation](@docroot@/glossary.md#gloss-store-derivation). Consult the [store chapter](@docroot@/store/derivation/index.md) for what a store derivation is; this section just concerns how to create one from the Nix language. This builtin function takes as input an attribute set, the attributes of which specify the inputs to the process. It outputs an attribute set, and produces a [store derivation](@docroot@/glossary.md#gloss-store-derivation) as a side effect of evaluation. ## Input attributes ### Required - \[`name`\]{#attr-name} ([String](@docroot@/language/types.md#type-string)) A symbolic name for the derivation. See [derivation outputs](@docroot@/store/derivation/outputs/index.md#outputs) for what this is affects. > **Example** > > ``` > derivation { > name = "hello"; > # ... > } > ``` > > The derivation's path will be `/nix/store/-hello.drv`. The [output](#attr-outputs) paths will be of the form `/nix/store/-hello[-]` - \[`system`\]{#attr-system} ([String](@docroot@/language/types.md#type-string)) See [system](@docroot@/store/derivation/index.md#system). > **Example** > > Declare a derivation to be built on a specific system type: > > ``` > derivation { > # ... > system = "x86_64-linux"; > # ... > } > ``` > **Example** > > Declare a derivation to be built on the system type that evaluates the expression: > > ``` > derivation { > # ... > system = builtins.currentSystem; > # ... > } > ``` > > [`builtins.currentSystem`](@docroot@/language/builtins.md#builtins-currentSystem) has the value of the \[`system` configuration option\], and defaults to the system type of the current Nix installation. - \[`builder`\]{#attr-builder} ([Path](@docroot@/language/types.md#type-path) | [String](@docroot@/language/types.md#type-string)) See [builder](@docroot@/store/derivation/index.md#builder). > **Example** > > Use the file located at `/bin/bash` as the builder executable: > > ``` > derivation { > # ... > builder = "/bin/bash"; > # ... > }; > ``` > **Example** > > Copy a local file to the Nix store for use as the builder executable: > > ``` > derivation { > # ... > builder = ./builder.sh; > # ... > }; > ``` > **Example** > > Use a file from another derivation as the builder executable: > > ``` > let pkgs = import {}; in > derivation { > # ... > builder = "${pkgs.python}/bin/python"; > # ... > }; > ``` ### Optional - \[`args`\]{#attr-args} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string)) Default: `[ ]` See [args](@docroot@/store/derivation/index.md#args). > **Example** > > Pass arguments to Bash to interpret a shell command: > > ``` > derivation { > # ... > builder = "/bin/bash"; > args = [ "-c" "echo hello world > $out" ]; > # ... > }; > ``` - \[`outputs`\]{#attr-outputs} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string)) Default: `[ "out" ]` Symbolic outputs of the derivation. Each output name is passed to the [`builder`](#attr-builder) executable as an environment variable with its value set to the corresponding [store path](@docroot@/store/store-path.md). By default, a derivation produces a single output called `out`. However, derivations can produce multiple outputs. This allows the associated [store objects](@docroot@/store/store-object.md) and their [closures](@docroot@/glossary.md#gloss-closure) to be copied or garbage-collected separately. > **Example** > > Imagine a library package that provides a dynamic library, header files, and documentation. A program that links against such a library doesn’t need the header files and documentation at runtime, and it doesn’t need the documentation at build time. Thus, the library package could specify: > > ``` > derivation { > # ... > outputs = [ "lib" "dev" "doc" ]; > # ... > } > ``` > > This will cause Nix to pass environment variables `lib`, `dev`, and `doc` to the builder containing the intended store paths of each output. The builder would typically do something like > > ``` > ./configure \ > --libdir=$lib/lib \ > --includedir=$dev/include \ > --docdir=$doc/share/doc > ``` > > for an Autoconf-style package. The name of an output is combined with the name of the derivation to create the [name part](@docroot@/store/store-path.md#name) of the output's store path, unless it is `out`, in which case just the name of the derivation is used. > **Example** > > ``` > derivation { > name = "example"; > outputs = [ "lib" "dev" "doc" "out" ]; > # ... > } > ``` > > The store derivation path will be `/nix/store/-example.drv`. The output paths will be > > - `/nix/store/-example-lib` > - `/nix/store/-example-dev` > - `/nix/store/-example-doc` > - `/nix/store/-example` You can refer to each output of a derivation by selecting it as an attribute. The first element of `outputs` determines the _default output_ and ends up at the top-level. > **Example** > > Select an output by attribute name: > > ``` > let > myPackage = derivation { > name = "example"; > outputs = [ "lib" "dev" "doc" "out" ]; > # ... > }; > in myPackage.dev > ``` > > Since `lib` is the first output, `myPackage` is equivalent to `myPackage.lib`. - See [Advanced Attributes](./advanced-attributes.md) for more, infrequently used, optional attributes. - Every other attribute is passed as an environment variable to the builder. Attribute values are translated to environment variables as follows: - Strings are passed unchanged. - Integral numbers are converted to decimal notation. - Floating point numbers are converted to simple decimal or scientific notation with a preset precision. - A _path_ (e.g., `../foo/sources.tar`) causes the referenced file to be copied to the store; its location in the store is put in the environment variable. The idea is that all sources should reside in the Nix store, since all inputs to a derivation should reside in the Nix store. - A _derivation_ causes that derivation to be built prior to the present derivation. The environment variable is set to the [store path](@docroot@/store/store-path.md) of the derivation's default [output](#attr-outputs). - Lists of the previous types are also allowed. They are simply concatenated, separated by spaces. - `true` is passed as the string `1`, `false` and `null` are passed as an empty string. ## [Evaluation](https://docharvest.github.io/docs/nix/language/evaluation/) Contents nix Evaluation Nix Evaluation Evaluation is the process of turning a Nix expression into a [Nix value](types.md). This happens by a number of rules, such as: - Constructing values from literals. For example the number literal `1` is turned into the number value `1`. - Applying operators For example the addition operator `+` is applied to two number values to produce a new number value. - Applying built-in functions For example the expression `builtins.isInt 1` is evaluated to `true`. - Applying user-defined functions For example the expression `(x: x + 1) 10` can[\*](#laziness) be thought of rewriting `x` in the function body to the argument, `10 + 1`, which is then evaluated to `11`. These rules are applied as needed, driven by the specific use of the expression. For example, this can occur in the Nix command line interface or interactively with the [repl (read-eval-print loop)](@docroot@/command-ref/new-cli/nix3-repl.md), which is a useful tool when learning about evaluation. # Details ## Values {#values} Nix values can be thought of as a subset of Nix expressions. For example, the expression `1 + 2` is not a value, because it can be reduced to `3`. The expression `3` is a value, because it cannot be reduced any further. Evaluation normally happens by applying rules to the "head" of the expression, which is the outermost part of the expression. The head of an expression like `[ 1 2 ]` is the list literal (`[ a1 a2 ]`), for `1 + 2` it is the addition operator (`+`), and for `f 1` it is the function application "operator" ( ). After applying all possible rules to the head until no rules can be applied, the expression is in "weak head normal form" (WHNF). This means that the outermost constructor of the expression is evaluated, but the inner values may or may not be. "Weak" only signifies that the expression may be a function. This is an historical or academic artifact, and Nix has no use for the non-weak "head normal form". ## Laziness and thunks {#laziness} The Nix language implements _call by need_ (as opposed to _call by value_ or _call by reference_). Call by need is commonly known as laziness in functional programming, as it is a specific implementation of the concept where evaluation is deferred until the result is required, aiming to only evaluate the parts of an expression that are needed to produce the final result. Furthermore, the result of evaluation is preserved, in values, in `let` bindings, in function _parameters_, which behave a lot like `let` bindings, but with the notable exception of function _calls_. Results of function calls rely on being put into `let` bindings, etc to be reused. When discussing the process of evaluation in lower level terms, we may define values not as a subset of expressions, but separately, where each "value" is either a data constructor, a function or a _thunk_. A thunk is a delayed computation, represented by an expression reference and a "closure" – the values for the lexical scope around the delayed expression. As a user of the language, you generally don't have to think about thunks, as they are not part of the language semantics, but you may encounter them in the repl, in the [C API](@docroot@/c-api.md) or in discussions. ## Strictness Instead of thinking about thunks, it is often more productive to think in terms of _strictness_. This term is used in functional programming to refer to the opposite of laziness, i.e. not just for something like error propagation. It refers to the need to evaluate certain expressions before evaluation can produce any result. Statements about strictness usually implicitly refer to weak head normal form. For example, we can say that the following function is strict in its argument: ``` x: isAttrs x || isFunction x ``` The above function must be strict in its argument `x` because determining its type requires evaluating `x` to at least some degree. The following function is not strict in its argument: ``` x: { isOk = isAttrs x || isFunction x; } ``` It is not strict, because it can return the attribute set before evaluating `x`. The attribute value for `isOk` _is_ strict in `x`. A function with a _set pattern_ is always strict in its argument, as a consequence of checking the argument's type and/or attribute names: ``` let f = { ... }: "ok"; in f (throw "kablam") => error: kablam ``` However, a set pattern does not add any strictness beyond WHNF of the attribute set argument. ``` let f = orig@{ x, ... }: "ok"; in f { x = throw "error"; y = throw "error"; } => "ok" ``` ## Evaluation order The order in which expressions are evaluated is generally unspecified, because it does not affect successful evaluation outcomes. This allows more freedom for the evaluator to evolve and to evaluate efficiently. Data dependencies naturally impose some ordering constraints: a value cannot be used before it is computed. Beyond these constraints, the evaluator is free to choose any order. The order in which side effects such as [`builtins.trace`](@docroot@/language/builtins.md#builtins-trace) output occurs is not defined, but may be expected to follow data dependencies. In a lazy language, evaluation order is often opposite to expectations from strict languages. For example, in `let wrap = x: { wrapped = x; }; in wrap (1 + 2)`, the function body produces a result (`{ wrapped = ...; }`) _before_ evaluating `x`. ## Infinite recursion and stack overflow During evaluation, two types of errors can occur when expressions reference themselves or call functions too deeply: ### Infinite recursion This error occurs when a value depends on itself through a cycle, making it impossible to compute. ``` let x = x; in x => error: infinite recursion encountered ``` Infinite recursion happens at the value level when evaluating an expression requires evaluating the same expression again. Despite the name, infinite recursion is cheap to compute and does not involve a stack overflow. The cycle is finite and fairly easy to detect. ### Stack overflow This error occurs when the call depth exceeds the maximum allowed limit. ``` let f = x: f (x + 1); in f 0 => error: stack overflow; max-call-depth exceeded ``` Stack overflow happens when too many function calls are nested without returning. The maximum call depth is controlled by the [`max-call-depth` setting](@docroot@/command-ref/conf-file.md#conf-max-call-depth). ## [Identifiers](https://docharvest.github.io/docs/nix/language/identifiers/) Contents nix Identifiers Nix Identifiers An _identifier_ is an [ASCII](https://en.wikipedia.org/wiki/ASCII) character sequence that: - Starts with a letter (`a-z`, `A-Z`) or underscore (`_`) - Can contain any number of: - Letters (`a-z`, `A-Z`) - Digits (`0-9`) - Underscores (`_`) - Apostrophes (`'`) - Hyphens (`-`) - Is not one of the [keywords](#keywords) > **Syntax** > > _identifier_ ~ `[A-Za-z_][A-Za-z0-9_'-]*` # Names A _name_ can be written as an [identifier](#identifiers) or a [string literal](./string-literals.md). > **Syntax** > > _name_ → _identifier_ | _string_ Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting-attributes). Two names are the same if they represent the same sequence of characters, regardless of whether they are written as identifiers or strings. # Keywords These keywords are reserved and cannot be used as [identifiers](#identifiers): - [`assert`](./syntax.md#assertions) - [`else`](./syntax.md#conditionals) - [`if`](./syntax.md#conditionals) - [`in`](./syntax.md#let-expressions) - [`inherit`](./syntax.md#inheriting-attributes) - [`let`](./syntax.md#let-expressions) - [`or`](./operators.md#attribute-selection) (see note) - [`rec`](./syntax.md#recursive-sets) - [`then`](./syntax.md#conditionals) - [`with`](./syntax.md#with-expressions) > **Note** > > The Nix language evaluator currently allows `or` to be used as a name in some contexts, for backwards compatibility reasons. Users are advised not to rely on this. > > There are long-standing issues with how `or` is parsed as a name, which can't be resolved without making a breaking change to the language. ## [Import From Derivation](https://docharvest.github.io/docs/nix/language/import-from-derivation/) Contents nix Import From Derivation Nix Import From Derivation The value of a Nix expression can depend on the contents of a [store object](@docroot@/store/store-object.md). Passing an expression `expr` that evaluates to a [store path](@docroot@/store/store-path.md) to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD): - [`import`](./builtins.md#builtins-import) `expr` - [`builtins.readFile`](./builtins.md#builtins-readFile) `expr` - [`builtins.readFileType`](./builtins.md#builtins-readFileType) `expr` - [`builtins.readDir`](./builtins.md#builtins-readDir) `expr` - [`builtins.pathExists`](./builtins.md#builtins-pathExists) `expr` - [`builtins.filterSource`](./builtins.md#builtins-filterSource) `f expr` - [`builtins.path`](./builtins.md#builtins-path) `{ path = expr; }` - [`builtins.hashFile`](./builtins.md#builtins-hashFile) `t expr` - `builtins.scopedImport x drv` When the store path needs to be accessed, evaluation will be paused, the corresponding store object [realised](@docroot@/glossary.md#gloss-realise), and then evaluation resumed. This has performance implications: Evaluation can only finish when all required store objects are realised. Since the Nix language evaluator is sequential, it only finds store paths to read from one at a time. While realisation is always parallel, in this case it cannot be done for all required store paths at once, and is therefore much slower than otherwise. Realising store objects during evaluation can be disabled by setting [`allow-import-from-derivation`](../command-ref/conf-file.md#conf-allow-import-from-derivation) to `false`. Without IFD it is ensured that evaluation is complete and Nix can produce a build plan before starting any realisation. ## Example In the following Nix expression, the inner derivation `drv` produces a file with contents `hello`. ``` # IFD.nix let drv = derivation { name = "hello"; builder = "/bin/sh"; args = [ "-c" "echo -n hello > $out" ]; system = builtins.currentSystem; }; in "${builtins.readFile drv} world" ``` ``` nix-instantiate IFD.nix --eval --read-write-mode ``` ``` building '/nix/store/348q1cal6sdgfxs8zqi9v8llrsn4kqkq-hello.drv'... "hello world" ``` The contents of the derivation's output have to be [realised](@docroot@/glossary.md#gloss-realise) before they can be read with [`readFile`](./builtins.md#builtins-readFile). Only then evaluation can continue to produce the final result. ## Illustration As a first approximation, the following data flow graph shows how evaluation and building are interleaved, if the value of a Nix expression depends on realising a [store object](@docroot@/store/store-object.md). Boxes are data structures, arrow labels are transformations. ``` +----------------------+ +------------------------+ | Nix evaluator | | Nix store | | .----------------. | | | | | Nix expression | | | | | '----------------' | | | | | | | | | evaluate | | | | | | | | | V | | | | .------------. | | | | | derivation | | | .------------------. | | | expression |----|-instantiate-|->| store derivation | | | '------------' | | '------------------' | | | | | | | | | realise | | | | | | | | | V | | .----------------. | | .--------------. | | | Nix expression |<-|----read-----|----| store object | | | '----------------' | | '--------------' | | | | | | | evaluate | | | | | | | | | V | | | | .------------. | | | | | value | | | | | '------------' | | | +----------------------+ +------------------------+ ``` In more detail, the following sequence diagram shows how the expression is evaluated step by step, and where evaluation is blocked to wait for the build output to appear. ``` .-------. .-------------. .---------. |Nix CLI| |Nix evaluator| |Nix store| '-------' '-------------' '---------' | | | |evaluate IFD.nix| | |--------------->| | | | | | evaluate `"${readFile drv} world"` | | | | | evaluate `readFile drv` | | | | | evaluate `drv` as string | | | | | |instantiate /nix/store/...-hello.drv| | |----------------------------------->| | : | | : realise /nix/store/...-hello.drv | | :----------------------------------->| | : | | |--------. | : | | | (evaluation blocked) | echo hello > $out | : | | | |<-------' | : /nix/store/...-hello | | |<-----------------------------------| | | | | resume `readFile /nix/store/...-hello` | | | | | | readFile /nix/store/...-hello | | |----------------------------------->| | | | | | hello | | |<-----------------------------------| | | | | resume `"${"hello"} world"` | | | | | resume `"hello world"` | | | | | "hello world" | | |<---------------| | .-------. .-------------. .---------. |Nix CLI| |Nix evaluator| |Nix store| '-------' '-------------' '---------' ``` ## [Operators](https://docharvest.github.io/docs/nix/language/operators/) Contents nix Operators Nix Operators Name Syntax Associativity Precedence [Attribute selection](#attribute-selection) _attrset_ `.` _attrpath_ \[ `or` _expr_ \] none 1 [Function application](#function-application) _func_ _expr_ left 2 [Arithmetic negation](#arithmetic) `-` _number_ none 3 [Has attribute](#has-attribute) _attrset_ `?` _attrpath_ none 4 List concatenation _list_ `++` _list_ right 5 [Multiplication](#arithmetic) _number_ `*` _number_ left 6 [Division](#arithmetic) _number_ `/` _number_ left 6 [Subtraction](#arithmetic) _number_ `-` _number_ left 7 [Addition](#arithmetic) _number_ `+` _number_ left 7 [String concatenation](#string-concatenation) _string_ `+` _string_ left 7 [Path concatenation](#path-concatenation) _path_ `+` _path_ left 7 [Path and string concatenation](#path-and-string-concatenation) _path_ `+` _string_ left 7 [String and path concatenation](#string-and-path-concatenation) _string_ `+` _path_ left 7 Logical negation (`NOT`) `!` _bool_ none 8 [Update](#update) _attrset_ `//` _attrset_ right 9 [Less than](#comparison) _expr_ `<` _expr_ none 10 [Less than or equal to](#comparison) _expr_ `<=` _expr_ none 10 [Greater than](#comparison) _expr_ `>` _expr_ none 10 [Greater than or equal to](#comparison) _expr_ `>=` _expr_ none 10 [Equality](#equality) _expr_ `==` _expr_ none 11 Inequality _expr_ `!=` _expr_ none 11 [Logical conjunction](#logical-conjunction) (`AND`) _bool_ `&&` _bool_ left [12](#precedence-and-disjunctive-normal-form) [Logical disjunction](#logical-disjunction) (`OR`) _bool_ `||` _bool_ left [13](#precedence-and-disjunctive-normal-form) [Logical implication](#logical-implication) _bool_ `->` _bool_ right 14 [Pipe operator](#pipe-operators) (experimental) _expr_ `|>` _func_ left 15 [Pipe operator](#pipe-operators) (experimental) _func_ `<|` _expr_ right 15 ## Attribute selection > **Syntax** > > _attrset_ `.` _attrpath_ \[ `or` _expr_ \] Select the attribute denoted by attribute path _attrpath_ from [attribute set](./types.md#type-attrs) _attrset_. If the attribute doesn’t exist, return the _expr_ after `or` if provided, otherwise abort evaluation. ## Function application > **Syntax** > > _func_ _expr_ Apply the callable value _func_ to the argument _expr_. Note the absence of any visible operator symbol. A callable value is either: - a [user-defined function](./syntax.md#functions) - a [built-in](./builtins.md) function - an attribute set with a [`__functor` attribute](./syntax.md#attr-__functor) > **Warning** > > [List](./types.md#type-list) items are also separated by whitespace, which means that function calls in list items must be enclosed by parentheses. ## Has attribute > **Syntax** > > _attrset_ `?` _attrpath_ Test whether [attribute set](./types.md#type-attrs) _attrset_ contains the attribute denoted by _attrpath_. The result is a [Boolean](./types.md#type-bool) value. See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr) After evaluating _attrset_ and _attrpath_, the computational complexity is O(log(_n_)) for _n_ attributes in the _attrset_ ## Arithmetic Numbers will retain their type unless mixed with other numeric types: Pure integer operations will always return integers, whereas any operation involving at least one floating point number returns a floating point number. Evaluation of the following numeric operations throws an evaluation error: - Division by zero - Integer overflow, that is, any operation yielding a result outside of the representable range of [Nix language integers](./syntax.md#number-literal) See also [Comparison](#comparison) and [Equality](#equality). The `+` operator is overloaded to also work on strings and paths. ## String concatenation > **Syntax** > > _string_ `+` _string_ Concatenate two [strings](./types.md#type-string) and merge their [string contexts](./string-context.md). ## Path concatenation > **Syntax** > > _path_ `+` _path_ Concatenate two [paths](./types.md#type-path). The result is a path. ## Path and string concatenation > **Syntax** > > _path_ + _string_ Concatenate _[path](./types.md#type-path)_ with _[string](./types.md#type-string)_. The result is a path. > **Note** > > The string must not have a [string context](./string-context.md) that refers to a [store path](@docroot@/store/store-path.md). ## String and path concatenation > **Syntax** > > _string_ + _path_ Concatenate _[string](./types.md#type-string)_ with _[path](./types.md#type-path)_. The result is a string. > **Important** > > The file or directory at _path_ must exist and is copied to the [store](@docroot@/glossary.md#gloss-store). The path appears in the result as the corresponding [store path](@docroot@/store/store-path.md). ## Update > **Syntax** > > _attrset1_ // _attrset2_ Update [attribute set](./types.md#type-attrs) _attrset1_ with names and values from _attrset2_. The returned attribute set will have all of the attributes in _attrset1_ and _attrset2_. If an attribute name is present in both, the attribute value from the latter is taken. This operator is [strict](@docroot@/language/evaluation.md#strictness) in both _attrset1_ and _attrset2_. That means that both arguments are evaluated to [weak head normal form](@docroot@/language/evaluation.md#values), so the attribute sets themselves are evaluated, but their attribute values are not evaluated. ## Comparison Comparison is - [arithmetic](#arithmetic) for [numbers](./types.md#type-float) - lexicographic for [strings](./types.md#type-string) and [paths](./types.md#type-path) - item-wise lexicographic for [lists](./types.md#type-list): elements at the same index in both lists are compared according to their type and skipped if they are equal. All comparison operators are implemented in terms of `<`, and the following equivalencies hold: comparison implementation _a_ `<=` _b_ `! (` _b_ `<` _a_ `)` _a_ `>` _b_ _b_ `<` _a_ _a_ `>=` _b_ `! (` _a_ `<` _b_ `)` ## Equality - [Attribute sets](./types.md#type-attrs) are compared first by attribute names and then by items until a difference is found. - [Lists](./types.md#type-list) are compared first by length and then by items until a difference is found. - Comparison of distinct [functions](./syntax.md#functions) returns `false`, but identical functions may be subject to [value identity optimization](#value-identity-optimization). - Numbers are type-compatible, see [arithmetic](#arithmetic) operators. - Floating point numbers only differ up to a limited precision. The `==` operator is [strict](@docroot@/language/evaluation.md#strictness) in both arguments; when comparing composite types ([attribute sets](./types.md#type-attrs) and [lists](./types.md#type-list)), it is partially strict in their contained values: they are evaluated until a difference is found. ### Value identity optimization Nix performs equality comparisons of nested values by pointer equality or more abstractly, _identity_. Nix semantics ideally do not assign a unique identity to values as they are created, but equality is an exception to this rule. The disputable benefit of this is that it is more efficient, and it allows cyclical structures to be compared, e.g. `let x = { x = x; }; in x == x` evaluates to `true`. However, as a consequence, it makes a function equal to itself when the comparison is made in a list or attribute set, in contradiction to a simple direct comparison. ## Logical conjunction > **Syntax** > > _bool1_ `&&` _bool2_ Logical AND. Equivalent to `if` _bool1_ `then` _bool2_ `else false`. This operator is [strict](@docroot@/language/evaluation.md#strictness) in _bool1_, but only evaluates _bool2_ if _bool1_ is `true`. > **Example** > > ``` > true && false > => false > > false && throw "never evaluated" > => false > ``` ## Logical disjunction > **Syntax** > > _bool1_ `||` _bool2_ Logical OR. Equivalent to `if` _bool1_ `then true` `else` _bool2_. This operator is [strict](@docroot@/language/evaluation.md#strictness) in _bool1_, but only evaluates _bool2_ if _bool1_ is `false`. > **Example** > > ``` > true || false > => true > > true || throw "never evaluated" > => true > ``` ### Precedence and disjunctive normal form The precedence of `&&` and `||` aligns with disjunctive normal form. Without parentheses, an expression describes multiple "permissible situations" (connected by `||`), where each situation consists of multiple simultaneous conditions (connected by `&&`). For example, `A || B && C || D && E` is parsed as `A || (B && C) || (D && E)`, describing three permissible situations: A holds, or both B and C hold, or both D and E hold. ## Logical implication > **Syntax** > > _bool1_ `->` _bool2_ Logical implication. Equivalent to `!`_bool1_ `||` _bool2_ (or `if` _bool1_ `then` _bool2_ `else true`). This operator is [strict](@docroot@/language/evaluation.md#strictness) in _bool1_, but only evaluates _bool2_ if _bool1_ is `true`. > **Example** > > ``` > true -> false > => false > > false -> throw "never evaluated" > => true > ``` ## Pipe operators - _a_ `|>` _b_ is equivalent to _b_ _a_ - _a_ `<|` _b_ is equivalent to _a_ _b_ > **Example** > > ``` > nix-repl> 1 |> builtins.add 2 |> builtins.mul 3 > 9 > > nix-repl> builtins.add 1 <| builtins.mul 2 <| 3 > 7 > ``` > **Warning** > > This syntax is part of an [experimental feature](@docroot@/development/experimental-features.md) and may change in future releases. > > To use this syntax, make sure the [`pipe-operators` experimental feature](@docroot@/development/experimental-features.md#xp-feature-pipe-operators) is enabled. For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md): > > ``` > extra-experimental-features = pipe-operators > ``` ## [Scoping rules](https://docharvest.github.io/docs/nix/language/scope/) Contents nix Scoping rules Nix Scoping rules A _scope_ in the Nix language is a dictionary keyed by [name](./identifiers.md#names), mapping each name to an expression and a _definition type_. The definition type is either _explicit_ or _implicit_. Each entry in this dictionary is a _definition_. Explicit definitions are created by the following expressions: - [let-expressions](syntax.md#let-expressions) - [recursive attribute set literals](syntax.md#recursive-sets) (`rec`) - [function literals](syntax.md#functions) Implicit definitions are only created by [with-expressions](./syntax.md#with-expressions). Every expression is _enclosed_ by a scope. The outermost expression is enclosed by the [built-in, global scope](./builtins.md), which contains only explicit definitions. The expressions listed above _extend_ their enclosing scope by adding new definitions, or replacing existing ones with the same name. An explicit definition can replace a definition of any type; an implicit definition can only replace another implicit definition. Each of the above expressions defines which of its subexpressions are enclosed by the extended scope. In all other cases, the same scope that encloses an expression is the enclosing scope for its subexpressions. The Nix language is [statically scoped](https://en.wikipedia.org/wiki/Scope_\(computer_science\)#Lexical_scope); the value of a variable is determined only by the variable's enclosing scope, and not by the dynamic context in which the variable is evaluated. > **Note** > > Expressions entered into the [Nix REPL](@docroot@/command-ref/new-cli/nix3-repl.md) are enclosed by a scope that can be extended by command line arguments or previous REPL commands. These ways of extending scope are not, strictly speaking, part of the Nix language. ## [String context](https://docharvest.github.io/docs/nix/language/string-context/) Contents nix String context Nix String context > **Note** > > This is an advanced topic. The Nix language is designed to be used without the programmer consciously dealing with string contexts or even knowing what they are. A string in the Nix language is not just a sequence of characters like strings in other languages. It is actually a pair of a sequence of characters and a _string context_. The string context is an (unordered) set of _string context elements_. The purpose of string contexts is to collect non-string values attached to strings via [string concatenation](./operators.md#string-concatenation), [string interpolation](./string-interpolation.md), and similar operations. The idea is that a user can reference other files when creating text files through Nix expressions, without manually keeping track of the exact paths. Nix will ensure that the all referenced files are accessible – that all [store paths](@docroot@/glossary.md#gloss-store-path) are [valid](@docroot@/glossary.md#gloss-validity). > **Note** > > String contexts are _not_ explicitly manipulated in idiomatic Nix language code. String context elements come in different forms: - [deriving path](@docroot@/glossary.md#gloss-deriving-path){#string-context-element-derived-path} A string context element of this type is a [deriving path](@docroot@/glossary.md#gloss-deriving-path). They can be either of type [constant](#string-context-constant) or [output](#string-context-output), which correspond to the types of deriving paths. - \[Constant string context elements\]{#string-context-constant} > **Example** > > [`builtins.storePath`](./builtins.md#builtins-storePath) creates a string with a single constant string context element: > > ``` > builtins.getContext (builtins.storePath "/nix/store/ikwkxz4wwlp2g1428n7dy729cg1d9hin-hello-2.10") > ``` > > evaluates to > > ``` > { > "/nix/store/ikwkxz4wwlp2g1428n7dy729cg1d9hin-hello-2.10" = { > path = true; > }; > } > ``` - \[Output string context elements\]{#string-context-output} > **Example** > > The behavior of string contexts are best demonstrated with a built-in function that is still experimental: [`builtins.outputOf`](./builtins.md#builtins-outputOf). This example will _not_ work with stable Nix! > > ``` > builtins.getContext > (builtins.outputOf > (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv") > "out") > ``` > > evaluates to > > ``` > { > "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = { > outputs = [ "out" ]; > }; > } > ``` - \[_derivation deep_\]{#string-context-element-derivation-deep} _derivation deep_ is an advanced feature intended to be used with the [`exportReferencesGraph` derivation attribute](./advanced-attributes.html#adv-attr-exportReferencesGraph). A _derivation deep_ string context element is a derivation path, and refers to both its outputs and the entire build closure of that derivation: all its outputs, all the other derivations the given derivation depends on, and all the outputs of those. > **Example** > > The best way to illustrate _derivation deep_ string contexts is with [`builtins.addDrvOutputDependencies`](./builtins.md#builtins-addDrvOutputDependencies). Take a regular constant string context element pointing to a derivation, and transform it into a "Derivation deep" string context element. > > ``` > builtins.getContext > (builtins.addDrvOutputDependencies > (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv")) > ``` > > evaluates to > > ``` > { > "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = { > allOutputs = true; > }; > } > ``` ## Inspecting string contexts Most basically, [`builtins.hasContext`](./builtins.md#builtins-hasContext) will tell whether a string has a non-empty context. When more granular information is needed, [`builtins.getContext`](./builtins.md#builtins-getContext) can be used. It creates an [attribute set](./types.md#type-attrs) representing the string context, which can be inspected as usual. ## Clearing string contexts [`builtins.unsafeDiscardStringContext`](./builtins.md#builtins-unsafeDiscardStringContext) will make a copy of a string, but with an empty string context. The returned string can be used in more ways, e.g. by operators that require the string context to be empty. The requirement to explicitly discard the string context in such use cases helps ensure that string context elements are not lost by mistake. The "unsafe" marker is only there to remind that Nix normally guarantees that dependencies are tracked, whereas the returned string has lost them. ## Constructing string contexts [`builtins.appendContext`](./builtins.md#builtins-appendContext) will create a copy of a string, but with additional string context elements. The context is specified explicitly by an [attribute set](./types.md#type-attrs) in the format that [`builtins.hasContext`](./builtins.md#builtins-hasContext) produces. A string with arbitrary contexts can be made like this: 1. Create a string with the desired string context elements. (The contents of the string do not matter.) 2. Dump its context with [`builtins.getContext`](./builtins.md#builtins-getContext). 3. Combine it with a base string and repeated [`builtins.appendContext`](./builtins.md#builtins-appendContext) calls. ## [String interpolation](https://docharvest.github.io/docs/nix/language/string-interpolation/) Contents nix String interpolation Nix String interpolation String interpolation is a language feature where a [string](./types.md#type-string), [path](./types.md#type-path), or [attribute name](./types.md#type-attrs) can contain expressions enclosed in `${ }` (dollar-sign with curly brackets). Such a construct is called _interpolated string_, and the expression inside is an [interpolated expression](#interpolated-expression). > **Syntax** > > _interpolation\_element_ → `${` _expression_ `}` ## Examples ### String Rather than writing ``` "--with-freetype2-library=" + freetype + "/lib" ``` (where `freetype` is a [derivation expression](@docroot@/glossary.md#gloss-derivation-expression)), you can instead write ``` "--with-freetype2-library=${freetype}/lib" ``` The latter is automatically translated to the former. A more complicated example (from the Nix expression for [Qt](http://www.trolltech.com/products/qt)): ``` configureFlags = " -system-zlib -system-libpng -system-libjpeg ${if openglSupport then "-dlopen-opengl -L${mesa}/lib -I${mesa}/include -L${libXmu}/lib -I${libXmu}/include" else ""} ${if threadSupport then "-thread" else "-no-thread"} "; ``` Note that Nix expressions and strings can be arbitrarily nested; in this case the outer string contains various interpolated expressions that themselves contain strings (e.g., `"-thread"`), some of which in turn contain interpolated expressions (e.g., `${mesa}`). To write a literal `${` in an regular string, escape it with a backslash (`\`). > **Example** > > ``` > "echo \${PATH}" > ``` > > ``` > "echo ${PATH}" > ``` To write a literal `${` in an indented string, escape it with two single quotes (`''`). > **Example** > > ``` > '' > echo ''${PATH} > '' > ``` > > ``` > "echo ${PATH}\n" > ``` `$${` can be written literally in any string. > **Example** > > In Make, `$` in file names or recipes is represented as `$$`, see [GNU `make`: Basics of Variable Reference](https://www.gnu.org/software/make/manual/html_node/Reference.html#Basics-of-Variable-References). This can be expressed directly in the Nix language strings: > > ``` > '' > MAKEVAR = Hello > all: > @export BASHVAR=world; echo $(MAKEVAR) $${BASHVAR} > '' > ``` > > ``` > "MAKEVAR = Hello\nall:\n\t@export BASHVAR=world; echo $(MAKEVAR) $\${BASHVAR}\n" > ``` See the [documentation on strings](./types.md#type-string) for details. ### Path Rather than writing ``` ./. + "/" + foo + "-" + bar + ".nix" ``` or ``` ./. + "/${foo}-${bar}.nix" ``` you can instead write ``` ./${foo}-${bar}.nix ``` ### Attribute name Attribute names can be interpolated strings. > **Example** > > ``` > let name = "foo"; in > { ${name} = 123; } > ``` > > ``` > { foo = 123; } > ``` Attributes can be selected with interpolated strings. > **Example** > > ``` > let name = "foo"; in > { foo = 123; }.${name} > ``` > > ``` > 123 > ``` # Interpolated expression An expression that is interpolated must evaluate to one of the following: - a [string](./types.md#type-string) - a [path](./types.md#type-path) - an [attribute set](./types.md#type-attrs) that has a `__toString` attribute or an `outPath` attribute - `__toString` must be a function that takes the attribute set itself and returns a string - `outPath` must be a string This includes [derivation expressions](./derivations.md) or [flake inputs](@docroot@/command-ref/new-cli/nix3-flake.md#flake-inputs) (experimental). A string interpolates to itself. A path in an interpolated expression is first copied into the Nix store, and the resulting string is the [store path](@docroot@/store/store-path.md) of the newly created [store object](@docroot@/store/store-object.md). > **Example** > > ``` > $ mkdir foo > ``` > > Reference the empty directory in an interpolated expression: > > ``` > "${./foo}" > ``` > > ``` > "/nix/store/2hhl2nz5v0khbn06ys82nrk99aa1xxdw-foo" > ``` A derivation interpolates to the [store path](@docroot@/store/store-path.md) of its first [output](./derivations.md#attr-outputs). > **Example** > > ``` > let > pkgs = import {}; > in > "${pkgs.hello}" > ``` > > ``` > "/nix/store/qnlr7906z0mrl2syrkdbpicffq02nw07-hello-2.12.1" > ``` An attribute set interpolates to the return value of the function in the `__toString` applied to the attribute set itself. > **Example** > > ``` > let > a = { > value = 1; > __toString = self: toString (self.value + 1); > }; > in > "${a}" > ``` > > ``` > "2" > ``` An attribute set also interpolates to the value of its `outPath` attribute. > **Example** > > ``` > let > a = { outPath = "foo"; }; > in > "${a}" > ``` > > ``` > "foo" > ``` If both `__toString` and `outPath` are present in an attribute set, `__toString` takes precedence. > **Example** > > ``` > let > a = { __toString = _: "yes"; outPath = throw "no"; }; > in > "${a}" > ``` > > ``` > "yes" > ``` If neither is present, an error is thrown. > **Example** > > ``` > let > a = {}; > in > "${a}" > ``` > > ``` > error: cannot coerce a set to a string: { } > > at «string»:4:2: > > 3| in > 4| "${a}" > | ^ > ``` ## [String literals](https://docharvest.github.io/docs/nix/language/string-literals/) Contents nix String literals Nix String literals A _string literal_ represents a [string](types.md#type-string) value. > **Syntax** > > _expression_ → _string_ > > _string_ → `"` ( _string\_char_\* [_interpolation\_element_](./string-interpolation.md) )\* _string\_char_\* `"` > > _string_ → `''` ( _indented\_string\_char_\* [_interpolation\_element_](./string-interpolation.md) )\* _indented\_string\_char_\* `''` > > _string_ → _uri_ > > _string\_char_ ~ `[^"$\\]|\$(?!\{)|\\.` > > _indented\_string\_char_ ~ `[^$']|\$\$|\$(?!\{)|''[$']|''\\.|'(?!')` > > _uri_ ~ `[A-Za-z][+\-.0-9A-Za-z]*:[!$%&'*+,\-./0-9:=?@A-Z_a-z~]+` Strings can be written in three ways. The most common way is to enclose the string between double quotes, e.g., `"foo bar"`. Strings can span multiple lines. The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation](./string-interpolation.md). The following must be escaped to represent them within a string, by prefixing with a backslash (`\`): - Double quote (`"`) > **Example** > > ``` > "\"" > ``` > > ``` > "\"" > ``` - Backslash (`\`) > **Example** > > ``` > "\\" > ``` > > ``` > "\\" > ``` - Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly" > **Example** > > ``` > "\${" > ``` > > ``` > "\${" > ``` The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively. A "double-dollar-curly" (`$${`) can be written literally. > **Example** > > ``` > "$${" > ``` > > ``` > "$\${" > ``` String values are output on the terminal with Nix-specific escaping. Strings written to files will contain the characters encoded by the escaping. The second way to write string literals is as an _indented string_, which is enclosed between pairs of _double single-quotes_ (`''`), like so: ``` '' This is the first line. This is the second line. This is the third line. '' ``` This kind of string literal intelligently strips indentation from the start of each line. To be precise, it strips from each line a number of spaces equal to the minimal indentation of the string as a whole (disregarding the indentation of empty lines). For instance, the first and second line are indented two spaces, while the third line is indented four spaces. Thus, two spaces are stripped from each line, so the resulting string is ``` "This is the first line.\nThis is the second line.\n This is the third line.\n" ``` > **Note** > > Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line. > **Warning** > > Prefixed tab characters are not stripped. > > > **Example** > > > > The following indented string is prefixed with tabs: > > > > ``` > > '' > > all: > > @echo hello > > '' > > ``` > > > > ``` > > "\tall:\n\t\t@echo hello\n" > > ``` Indented strings support [string interpolation](./string-interpolation.md). The following must be escaped to represent them in an indented string: - `$` is escaped by prefixing it with two single quotes (`''`) > **Example** > > ``` > '' > ''$ > '' > ``` > > ``` > "$\n" > ``` - `''` is escaped by prefixing it with one single quote (`'`) > **Example** > > ``` > '' > ''' > '' > ``` > > ``` > "''\n" > ``` These special characters are escaped as follows: - Linefeed (`\n`): `''\n` - Carriage return (`\r`): `''\r` - Tab (`\t`): `''\t` `''\` escapes any other character. A "dollar-curly" (`${`) can be written as follows: > **Example** > > ``` > '' > echo ''${PATH} > '' > ``` > > ``` > "echo ${PATH}\n" > ``` > **Note** > > This differs from the syntax for escaping a dollar-curly within double quotes (`"\${"`). Be aware of which one is needed at a given moment. A "double-dollar-curly" (`$${`) can be written literally. > **Example** > > ``` > '' > $${ > '' > ``` > > ``` > "$\${\n" > ``` Indented strings are primarily useful in that they allow multi-line string literals to follow the indentation of the enclosing Nix expression, and that less escaping is typically necessary for strings representing languages such as shell scripts and configuration files because `''` is much less common than `"`. Example: ``` stdenv.mkDerivation { ... postInstall = '' mkdir $out/bin $out/etc cp foo $out/bin echo "Hello World" > $out/etc/foo.conf ${if enableBar then "cp bar $out/bin" else ""} ''; ... } ``` Finally, as a convenience, _URIs_ as defined in appendix B of [RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written _as is_, without quotes. For instance, the string `"http://example.org/foo.tar.bz2"` can also be written as `http://example.org/foo.tar.bz2`. ## [Language Constructs](https://docharvest.github.io/docs/nix/language/syntax/) Contents nix Language Constructs Nix Language Constructs This section covers syntax and semantics of the Nix language. ## Basic Literals ### String {#string-literal} See [String literals](string-literals.md). ### Number {#number-literal} Numbers, which can be _integers_ (like `123`) or _floating point_ (like `123.43` or `.27e13`). Integers in the Nix language are 64-bit [two's complement](https://en.wikipedia.org/wiki/Two%27s_complement) signed integers, with a range of -9223372036854775808 to 9223372036854775807, inclusive. Note that negative numeric literals are actually parsed as unary negation of positive numeric literals. This means that the minimum integer `-9223372036854775808` cannot be written as-is as a literal, since the positive number `9223372036854775808` is one past the maximum range. See [arithmetic](./operators.md#arithmetic) and [comparison](./operators.md#comparison) operators for semantics. ### Path {#path-literal} _Paths_ can be expressed by path literals such as `./builder.sh`. A path literal must contain at least one slash to be recognised as such. For instance, `builder.sh` is not a path: it's parsed as an expression that selects the attribute `sh` from the variable `builder`. Path literals are resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory). Path literals may also refer to absolute paths by starting with a slash. > **Note** > > Absolute paths make expressions less portable. In the case where a function translates a path literal into an absolute path string for a configuration file, it is recommended to write a string literal instead. This avoids some confusion about whether files at that location will be used during evaluation. It also avoids unintentional situations where some function might try to copy everything at the location into the store. If the first component of a path is a `~`, it is interpreted such that the rest of the path were relative to the user's home directory. For example, `~/foo` would be equivalent to `/home/edolstra/foo` for a user whose home directory is `/home/edolstra`. Path literals that start with `~` are not allowed in [pure](@docroot@/command-ref/conf-file.md#conf-pure-eval) evaluation. Path literals can also include [string interpolation](./string-interpolation.md), besides being [interpolated into other expressions](./string-interpolation.md#interpolated-expression). At least one slash (`/`) must appear _before_ any interpolated expression for the result to be recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid number division operation. `./a.${foo}/b.${bar}` is a path. [Lookup path](./constructs/lookup-path.md) literals such as `` also resolve to path values. ## List {#list-literal} Lists are formed by enclosing a whitespace-separated list of values between square brackets. For example, ``` [ 123 ./foo.nix "abc" (f { x = y; }) ] ``` defines a list of four elements, the last being the result of a call to the function `f`. Note that function calls have to be enclosed in parentheses. If they had been omitted, e.g., ``` [ 123 ./foo.nix "abc" f { x = y; } ] ``` the result would be a list of five elements, the fourth one being a function and the fifth being a set. Note that lists are only lazy in values, and they are strict in length. Elements in a list can be accessed using [`builtins.elemAt`](./builtins.md#builtins-elemAt). ## Attribute Set {#attrs-literal} An attribute set is a collection of name-value-pairs called _attributes_. Attribute sets are written enclosed in curly brackets (`{ }`). Attribute names and attribute values are separated by an equal sign (`=`). Each value can be an arbitrary expression, terminated by a semicolon (`;`) An attribute name is a string without context, and is denoted by a [name](./identifiers.md#names) (an [identifier](./identifiers.md#identifiers) or [string literal](string-literals.md)). > **Syntax** > > _attrset_ → `{` { _name_ `=` _expr_ `;` } `}` Attributes can appear in any order. An attribute name may only occur once in each attribute set. > **Example** > > This defines an attribute set with attributes named: > > - `x` with the value `123`, an integer > - `text` with the value `"Hello"`, a string > - `y` where the value is the result of applying the function `f` to the attribute set `{ bla = 456; }` > > ``` > { > x = 123; > text = "Hello"; > y = f { bla = 456; }; > } > ``` Attributes in nested attribute sets can be written using _attribute paths_. > **Syntax** > > _attrset_ → `{` { _attrpath_ `=` _expr_ `;` } `}` An attribute path is a dot-separated list of [names](./identifiers.md#names). > **Syntax** > > _attrpath_ = _name_ { `.` _name_ } > **Example** > > ``` > { a.b.c = 1; a.b.d = 2; } > ``` > > ``` > { > a = { > b = { > c = 1; > d = 2; > }; > }; > } > ``` Attribute names can also be set implicitly by using the [`inherit` keyword](#inheriting-attributes). > **Example** > > ``` > { inherit (builtins) true; } > ``` > > ``` > { true = true; } > ``` Attributes can be accessed with the [`.` operator](./operators.md#attribute-selection). Example: ``` { a = "Foo"; b = "Bar"; }.a ``` This evaluates to `"Foo"`. It is possible to provide a default value in an attribute selection using the `or` keyword. Example: ``` { a = "Foo"; b = "Bar"; }.c or "Xyzzy" ``` ``` { a = "Foo"; b = "Bar"; }.c.d.e.f.g or "Xyzzy" ``` will both evaluate to `"Xyzzy"` because there is no `c` attribute in the set. You can use arbitrary double-quoted strings as attribute names: ``` { "$!@#?" = 123; }."$!@#?" ``` ``` let bar = "bar"; in { "foo ${bar}" = 123; }."foo ${bar}" ``` Both will evaluate to `123`. Attribute names support [string interpolation](./string-interpolation.md): ``` let bar = "foo"; in { foo = 123; }.${bar} ``` ``` let bar = "foo"; in { ${bar} = 123; }.foo ``` Both will evaluate to `123`. In the special case where an attribute name inside of a set declaration evaluates to `null` (which is normally an error, as `null` cannot be coerced to a string), that attribute is simply not added to the set: ``` { ${if foo then "bar" else null} = true; } ``` This will evaluate to `{}` if `foo` evaluates to `false`. A set that has a \[`__functor`\]{#attr-\_\_functor} attribute whose value is callable (i.e. is itself a function or a set with a `__functor` attribute whose value is callable) can be applied as if it were a function, with the set itself passed in first , e.g., ``` let add = { __functor = self: x: x + self.x; }; inc = add // { x = 1; }; # inc is { x = 1; __functor = (...) } in inc 1 # equivalent of `add.__functor add 1` i.e. `1 + self.x` ``` evaluates to `2`. This can be used to attach metadata to a function without the caller needing to treat it specially, or to implement a form of object-oriented programming, for example. ## Recursive sets Recursive sets are like normal [attribute sets](./types.md#type-attrs), but the attributes can refer to each other. > _rec-attrset_ = `rec {` \[ _name_ `=` _expr_ `;` `]`... `}` Example: ``` rec { x = y; y = 123; }.x ``` This evaluates to `123`. Note that without `rec` the binding `x = y;` would refer to the variable `y` in the surrounding scope, if one exists, and would be invalid if no such variable exists. That is, in a normal (non-recursive) set, attributes are not added to the lexical scope; in a recursive set, they are. Recursive sets of course introduce the danger of infinite recursion. For example, the expression ``` rec { x = y; y = x; }.x ``` will crash with an `infinite recursion encountered` error message. ## Let-expressions A let-expression allows you to define local variables for an expression. > _let-in_ = `let` \[ _identifier_ = _expr_ `;` \]... `in` _expr_ Example: ``` let x = "foo"; y = "bar"; in x + y ``` This evaluates to `"foobar"`. There is also another, older, syntax for let expressions that should not be used in new code: > _let_ = `let` `{` _identifier_ = _expr_ `;` \[ _identifier_ = _expr_ `;`\]... `}` In this form, the attribute set between the `{` `}` is recursive. One of the attributes must have the special name `body`, which is the result of the expression. Example: ``` let { foo = bar; bar = "baz"; body = foo; } ``` This evaluates to "baz". ## Inheriting attributes When defining an [attribute set](./types.md#type-attrs) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes). This can be shortened using the `inherit` keyword. Example: ``` let x = 123; in { inherit x; y = 456; } ``` is equivalent to ``` let x = 123; in { x = x; y = 456; } ``` and both evaluate to `{ x = 123; y = 456; }`. > **Note** > > This works because `x` is added to the lexical scope by the `let` construct. It is also possible to inherit attributes from another attribute set. Example: In this fragment from `all-packages.nix`, ``` graphviz = (import ../tools/graphics/graphviz) { inherit fetchurl stdenv libpng libjpeg expat x11 yacc; inherit (xorg) libXaw; }; xorg = { libX11 = ...; libXaw = ...; ... } libpng = ...; libjpg = ...; ... ``` the set used in the function call to the function defined in `../tools/graphics/graphviz` inherits a number of variables from the surrounding scope (`fetchurl` ... `yacc`), but also inherits `libXaw` (the X Athena Widgets) from the `xorg` set. Summarizing the fragment ``` ... inherit x y z; inherit (src-set) a b c; ... ``` is equivalent to ``` ... x = x; y = y; z = z; a = src-set.a; b = src-set.b; c = src-set.c; ... ``` when used while defining local variables in a let-expression or while defining a set. In a `let` expression, `inherit` can be used to selectively bring specific attributes of a set into scope. For example ``` let x = { a = 1; b = 2; }; inherit (builtins) attrNames; in { names = attrNames x; } ``` is equivalent to ``` let x = { a = 1; b = 2; }; in { names = builtins.attrNames x; } ``` both evaluate to `{ names = [ "a" "b" ]; }`. ## Functions Functions have the following form: ``` pattern: body ``` The pattern specifies what the argument of the function must look like, and binds variables in the body to (parts of) the argument. There are three kinds of patterns: - If a pattern is a single identifier, then the function matches any argument. Example: ``` let negate = x: !x; concat = x: y: x + y; in if negate true then concat "foo" "bar" else "" ``` Note that `concat` is a function that takes one argument and returns a function that takes another argument. This allows partial parameterisation (i.e., only filling some of the arguments of a function); e.g., ``` map (concat "foo") [ "bar" "bla" "abc" ] ``` evaluates to `[ "foobar" "foobla" "fooabc" ]`. - A _set pattern_ of the form `{ name1, name2, …, nameN }` matches a set containing the listed attributes, and binds the values of those attributes to variables in the function body. For example, the function ``` { x, y, z }: z + y + x ``` can only be called with a set containing exactly the attributes `x`, `y` and `z`. No other attributes are allowed. If you want to allow additional arguments, you can use an ellipsis (`...`): ``` { x, y, z, ... }: z + y + x ``` This works on any set that contains at least the three named attributes. - It is possible to provide _default values_ for attributes, in which case they are allowed to be missing. A default value is specified by writing `name ? e`, where _e_ is an arbitrary expression. For example, ``` { x, y ? "foo", z ? "bar" }: z + y + x ``` specifies a function that only requires an attribute named `x`, but optionally accepts `y` and `z`. - An `@`\-pattern provides a means of referring to the whole value being matched: ``` args@{ x, y, z, ... }: z + y + x + args.a ``` but can also be written as: ``` { x, y, z, ... } @ args: z + y + x + args.a ``` Here `args` is bound to the argument _as passed_, which is further matched against the pattern `{ x, y, z, ... }`. The `@`\-pattern makes mainly sense with an ellipsis(`...`) as you can access attribute names as `a`, using `args.a`, which was given as an additional attribute to the function. > **Warning** > > `args@` binds the name `args` to the attribute set that is passed to the function. In particular, `args` does _not_ include any default values specified with `?` in the function's set pattern. > > For instance > > ``` > let > f = args@{ a ? 23, ... }: [ a args ]; > in > f {} > ``` > > is equivalent to > > ``` > let > f = args @ { ... }: [ (args.a or 23) args ]; > in > f {} > ``` > > and both expressions will evaluate to: > > ``` > [ 23 {} ] > ``` - All bindings introduced by the function are in scope in the entire function expression; not just in the body. It can therefore be used in default values. > **Example** > > A parameter (`x`), is used in the default value for another parameter (`y`): > > ``` > let > f = { x, y ? [x] }: { inherit y; }; > in > f { x = 3; } > ``` > > This evaluates to: > > ``` > { > y = [ 3 ]; > } > ``` > **Example** > > The binding of an `@` pattern, `args`, is used in the default value for a parameter, `x`: > > ``` > let > f = args@{ x ? args.a, ... }: x; > in > f { a = 1; } > ``` > > This evaluates to: > > ``` > 1 > ``` Note that functions do not have names. If you want to give them a name, you can bind them to an attribute, e.g., ``` let concat = { x, y }: x + y; in concat { x = "foo"; y = "bar"; } ``` ## Conditionals Conditionals look like this: ``` if e1 then e2 else e3 ``` where _e1_ is an expression that should evaluate to a Boolean value (`true` or `false`). ## Assertions Assertions are generally used to check that certain requirements on or between features and dependencies hold. They look like this: ``` assert e1; e2 ``` where _e1_ is an expression that should evaluate to a Boolean value. If it evaluates to `true`, _e2_ is returned; otherwise expression evaluation is aborted and a backtrace is printed. Here is a Nix expression for the Subversion package that shows how assertions can be used:. ``` { localServer ? false , httpServer ? false , sslSupport ? false , pythonBindings ? false , javaSwigBindings ? false , javahlBindings ? false , stdenv, fetchurl , openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null }: assert localServer -> db4 != null; ① assert httpServer -> httpd != null && httpd.expat == expat; ② assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); ③ assert pythonBindings -> swig != null && swig.pythonSupport; assert javaSwigBindings -> swig != null && swig.javaSupport; assert javahlBindings -> j2sdk != null; stdenv.mkDerivation { name = "subversion-1.1.1"; ... openssl = if sslSupport then openssl else null; ④ ... } ``` The points of interest are: 1. This assertion states that if Subversion is to have support for local repositories, then Berkeley DB is needed. So if the Subversion function is called with the `localServer` argument set to `true` but the `db4` argument set to `null`, then the evaluation fails. Note that `->` is the [logical implication](https://en.wikipedia.org/wiki/Truth_table#Logical_implication) Boolean operation. 2. This is a more subtle condition: if Subversion is built with Apache (`httpServer`) support, then the Expat library (an XML library) used by Subversion should be same as the one used by Apache. This is because in this configuration Subversion code ends up being linked with Apache code, and if the Expat libraries do not match, a build- or runtime link error or incompatibility might occur. 3. This assertion says that in order for Subversion to have SSL support (so that it can access `https` URLs), an OpenSSL library must be passed. Additionally, it says that _if_ Apache support is enabled, then Apache's OpenSSL should match Subversion's. (Note that if Apache support is not enabled, we don't care about Apache's OpenSSL.) 4. The conditional here is not really related to assertions, but is worth pointing out: it ensures that if SSL support is disabled, then the Subversion derivation is not dependent on OpenSSL, even if a non-`null` value was passed. This prevents an unnecessary rebuild of Subversion if OpenSSL changes. ## With-expressions A _with-expression_, ``` with e1; e2 ``` introduces the set _e1_ into the lexical scope of the expression _e2_. For instance, ``` let as = { x = "foo"; y = "bar"; }; in with as; x + y ``` evaluates to `"foobar"` since the `with` adds the `x` and `y` attributes of `as` to the lexical scope in the expression `x + y`. The most common use of `with` is in conjunction with the `import` function. E.g., ``` with (import ./definitions.nix); ... ``` makes all attributes defined in the file `definitions.nix` available as if they were defined locally in a `let`\-expression. The bindings introduced by `with` do not shadow bindings introduced by other means, e.g. ``` let a = 3; in with { a = 1; }; let a = 4; in with { a = 2; }; ... ``` establishes the same scope as ``` let a = 1; in let a = 2; in let a = 3; in let a = 4; in ... ``` Variables coming from outer `with` expressions _are_ shadowed: ``` with { a = "outer"; }; with { a = "inner"; }; a ``` Does evaluate to `"inner"`. ## Comments - Inline comments start with `#` and run until the end of the line. > **Example** > > ``` > # A number > 2 # Equals 1 + 1 > ``` > > ``` > 2 > ``` - Block comments start with `/*` and run until the next occurrence of `*/`. > **Example** > > ``` > /* > Block comments > can span multiple lines. > */ "hello" > ``` > > ``` > "hello" > ``` This means that block comments cannot be nested. > **Example** > > ``` > /* /* nope */ */ 1 > ``` > > ``` > error: syntax error, unexpected '*' > > at «string»:1:15: > > 1| /* /* nope */ * > | ^ > ``` Consider escaping nested comments and unescaping them in post-processing. > **Example** > > ``` > /* /* nested *\/ */ 1 > ``` > > ``` > 1 > ``` ## [Data Types](https://docharvest.github.io/docs/nix/language/types/) Contents nix Data Types Nix Data Types Every value in the Nix language has one of the following types: - [Integer](#type-int) - [Float](#type-float) - [Boolean](#type-bool) - [String](#type-string) - [Path](#type-path) - [Null](#type-null) - [Attribute set](#type-attrs) - [List](#type-list) - [Function](#type-function) - [External](#type-external) ## Primitives ### Integer {#type-int} An _integer_ in the Nix language is a signed 64-bit integer. Non-negative integers can be expressed as [integer literals](syntax.md#number-literal). Negative integers are created with the [arithmetic negation operator](./operators.md#arithmetic). The function [`builtins.isInt`](builtins.md#builtins-isInt) can be used to determine if a value is an integer. ### Float {#type-float} A _float_ in the Nix language is a 64-bit [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) floating-point number. Most non-negative floats can be expressed as [float literals](syntax.md#number-literal). Negative floats are created with the [arithmetic negation operator](./operators.md#arithmetic). The function [`builtins.isFloat`](builtins.md#builtins-isFloat) can be used to determine if a value is a float. ### Boolean {#type-bool} A _boolean_ in the Nix language is one of _true_ or _false_. These values are available as attributes of [`builtins`](builtins.md#builtins-builtins) as [`builtins.true`](builtins.md#builtins-true) and [`builtins.false`](builtins.md#builtins-false). The function [`builtins.isBool`](builtins.md#builtins-isBool) can be used to determine if a value is a boolean. ### String {#type-string} A _string_ in the Nix language is an immutable, finite-length sequence of bytes, along with a [string context](string-context.md). Nix does not assume or support working natively with character encodings. String values without string context can be expressed as [string literals](string-literals.md). The function [`builtins.isString`](builtins.md#builtins-isString) can be used to determine if a value is a string. ### Path {#type-path} A _path_ in the Nix language is an immutable, finite-length sequence of bytes starting with `/`, representing a POSIX-style, canonical file system path. Path values are distinct from string values, even if they contain the same sequence of bytes. Operations that produce paths will simplify the result as the standard C function [`realpath`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html) would, except that there is no symbolic link resolution. Paths are suitable for referring to local files, and are often preferable over strings. - Path values do not contain trailing or duplicate slashes, `.`, or `..`. - Relative path literals are automatically resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory). - Tooling can recognize path literals and provide additional features, such as autocompletion, refactoring automation and jump-to-file. A file is not required to exist at a given path in order for that path value to be valid, but a path that is converted to a string with [string interpolation](string-interpolation.md#interpolated-expression) or [string-and-path concatenation](operators.md#string-and-path-concatenation) must resolve to a readable file or directory which will be copied into the Nix store. For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/-foo.txt"`. Operations such as [`import`](builtins.md#builtins-import) can also expect a path to resolve to a readable file or directory. > **Note** > > The Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression. For example, assume you used a file path in an interpolated string during a `nix repl` session. Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path](@docroot@/store/store-path.md), since Nix might not re-read the file contents. Use `:r` to reset the repl as needed. Path values can be expressed as [path literals](syntax.md#path-literal). The function [`builtins.isPath`](builtins.md#builtins-isPath) can be used to determine if a value is a path. ### Null {#type-null} There is a single value of type _null_ in the Nix language. This value is available as an attribute on the [`builtins`](builtins.md#builtins-builtins) attribute set as [`builtins.null`](builtins.md#builtins-null). ## Compound values ### Attribute set {#type-attrs} An attribute set can be constructed with an [attribute set literal](syntax.md#attrs-literal). The function [`builtins.isAttrs`](builtins.md#builtins-isAttrs) can be used to determine if a value is an attribute set. ### List {#type-list} A list can be constructed with a [list literal](syntax.md#list-literal). The function [`builtins.isList`](builtins.md#builtins-isList) can be used to determine if a value is a list. ## Function {#type-function} A function can be constructed with a [function expression](syntax.md#functions). The function [`builtins.isFunction`](builtins.md#builtins-isFunction) can be used to determine if a value is a function. ## External {#type-external} An _external_ value is an opaque value created by a Nix [plugin](../command-ref/conf-file.md#conf-plugin-files). Such a value can be substituted in Nix expressions but only created and used by plugin code. ## [Data Types](https://docharvest.github.io/docs/nix/language/values/) Contents nix Data Types Nix Data Types ## [Variables](https://docharvest.github.io/docs/nix/language/variables/) Contents nix Variables Nix Variables A _variable_ is an [identifier](identifiers.md) used as an expression. > **Syntax** > > _expression_ → _identifier_ A variable must have the same name as a definition in the [scope](./scope.md) that encloses it. The value of a variable is the value of the corresponding expression in the enclosing scope. ## [Package Management](https://docharvest.github.io/docs/nix/package-management/) Contents nix Package Management Nix Package Management This chapter discusses how to do package management with Nix, i.e., how to obtain, install, upgrade, and erase packages. This is the “user’s” perspective of the Nix system — people who want to _create_ packages should consult the chapter on the [Nix language](../language/index.md). ## [Serving a Nix store via HTTP](https://docharvest.github.io/docs/nix/package-management/binary-cache-substituter/) Contents nix Serving a Nix store via HTTP Nix Serving a Nix store via HTTP You can easily share the Nix store of a machine via HTTP. This allows other machines to fetch store paths from that machine to speed up installations. It uses the same _binary cache_ mechanism that Nix usually uses to fetch pre-built binaries from [https://cache.nixos.org](https://cache.nixos.org). The daemon that handles binary cache requests via HTTP, `nix-serve`, is not part of the Nix distribution, but you can install it from Nixpkgs: ``` $ nix-env --install --attr nixpkgs.nix-serve ``` You can then start the server, listening for HTTP connections on whatever port you like: ``` $ nix-serve -p 8080 ``` To check whether it works, try fetching the [`nix-cache-info`](@docroot@/protocols/binary-cache/nix-cache-info.md) file on the client: ``` $ curl http://avalon:8080/nix-cache-info StoreDir: /nix/store WantMassQuery: 1 Priority: 30 ``` When writing to a binary cache (e.g., with [`nix copy`](@docroot@/command-ref/new-cli/nix3-copy.md)), Nix creates [`nix-cache-info`](@docroot@/protocols/binary-cache/nix-cache-info.md) automatically if it doesn't exist. On the client side, you can tell Nix to use your binary cache using `--substituters`, e.g.: ``` $ nix-env --install --attr nixpkgs.firefox --substituters http://avalon:8080/ ``` The option `substituters` tells Nix to use this binary cache in addition to your default caches, such as [https://cache.nixos.org](https://cache.nixos.org). Thus, for any path in the closure of Firefox, Nix will first check if the path is available on the server `avalon` or another binary caches. If not, it will fall back to building from source. You can also tell Nix to always use your binary cache by adding a line to the `nix.conf` configuration file like this: ``` substituters = http://avalon:8080/ https://cache.nixos.org/ ``` ## [Garbage Collection](https://docharvest.github.io/docs/nix/package-management/garbage-collection/) Contents nix Garbage Collection Nix Garbage Collection `nix-env` operations such as upgrades (`-u`) and uninstall (`-e`) never actually delete packages from the system. All they do (as shown above) is to create a new user environment that no longer contains symlinks to the “deleted” packages. Of course, since disk space is not infinite, unused packages should be removed at some point. You can do this by running the Nix garbage collector. It will remove from the Nix store any package not used (directly or indirectly) by any generation of any profile. Note however that as long as old generations reference a package, it will not be deleted. After all, we wouldn’t be able to do a rollback otherwise. So in order for garbage collection to be effective, you should also delete (some) old generations. Of course, this should only be done if you are certain that you will not need to roll back. To delete all old (non-current) generations of your current profile: ``` $ nix-env --delete-generations old ``` Instead of `old` you can also specify a list of generations, e.g., ``` $ nix-env --delete-generations 10 11 14 ``` To delete all generations older than a specified number of days (except the current generation), use the `d` suffix. For example, ``` $ nix-env --delete-generations 14d ``` deletes all generations older than two weeks. After removing appropriate old generations you can run the garbage collector as follows: ``` $ nix-store --gc ``` The behaviour of the garbage collector is affected by the `keep-derivations` (default: true) and `keep-outputs` (default: false) options in the Nix configuration file. The defaults will ensure that all derivations that are build-time dependencies of garbage collector roots will be kept and that all output paths that are runtime dependencies will be kept as well. All other derivations or paths will be collected. (This is usually what you want, but while you are developing it may make sense to keep outputs to ensure that rebuild times are quick.) If you are feeling uncertain, you can also first view what files would be deleted: ``` $ nix-store --gc --print-dead ``` Likewise, the option `--print-live` will show the paths that _won’t_ be deleted. There is also a convenient little utility `nix-collect-garbage`, which when invoked with the `-d` (`--delete-old`) switch deletes all old generations of all profiles in `/nix/var/nix/profiles`. So ``` $ nix-collect-garbage -d ``` is a quick and easy way to clean up your system. ## [Garbage Collector Roots](https://docharvest.github.io/docs/nix/package-management/garbage-collector-roots/) Contents nix Garbage Collector Roots Nix Garbage Collector Roots The roots of the garbage collector are all store paths to which there are symlinks in the directory `prefix/nix/var/nix/gcroots`. For instance, the following command makes the path `/nix/store/d718ef...-foo` a root of the collector: ``` $ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar ``` That is, after this command, the garbage collector will not remove `/nix/store/d718ef...-foo` or any of its dependencies. Subdirectories of `prefix/nix/var/nix/gcroots` are searched recursively. Symlinks to store paths count as roots. Symlinks to non-store paths are ignored, unless the non-store path is itself a symlink to a store path. ## [Profiles](https://docharvest.github.io/docs/nix/package-management/profiles/) Contents nix Profiles Nix Profiles Profiles and user environments are Nix’s mechanism for implementing the ability to allow different users to have different configurations, and to do atomic upgrades and rollbacks. To understand how they work, it’s useful to know a bit about how Nix works. In Nix, packages are stored in unique locations in the _Nix store_ (typically, `/nix/store`). For instance, a particular version of the Subversion package might be stored in a directory `/nix/store/dpmvp969yhdqs7lm2r1a3gng7pyq6vy4-subversion-1.1.3/`, while another version might be stored in `/nix/store/5mq2jcn36ldlmh93yj1n8s9c95pj7c5s-subversion-1.1.2`. The long strings prefixed to the directory names are cryptographic hashes (to be precise, 160-bit truncations of SHA-256 hashes encoded in [Nix32](@docroot@/protocols/nix32.md)) of _all_ inputs involved in building the package — sources, dependencies, compiler flags, and so on. So if two packages differ in any way, they end up in different locations in the file system, so they don’t interfere with each other. Here is what a part of a typical Nix store looks like: Of course, you wouldn’t want to type ``` $ /nix/store/dpmvp969yhdq...-subversion-1.1.3/bin/svn ``` every time you want to run Subversion. Of course we could set up the `PATH` environment variable to include the `bin` directory of every package we want to use, but this is not very convenient since changing `PATH` doesn’t take effect for already existing processes. The solution Nix uses is to create directory trees of symlinks to _activated_ packages. These are called _user environments_ and they are packages themselves (though automatically generated by `nix-env`), so they too reside in the Nix store. For instance, in the figure above, the user environment `/nix/store/0c1p5z4kda11...-user-env` contains a symlink to just Subversion 1.1.2 (arrows in the figure indicate symlinks). This would be what we would obtain if we had done ``` $ nix-env --install --attr nixpkgs.subversion ``` on a set of Nix expressions that contained Subversion 1.1.2. This doesn’t in itself solve the problem, of course; you wouldn’t want to type `/nix/store/0c1p5z4kda11...-user-env/bin/svn` either. That’s why there are symlinks outside of the store that point to the user environments in the store; for instance, the symlinks `default-42-link` and `default-43-link` in the example. These are called _generations_ since every time you perform a `nix-env` operation, a new user environment is generated based on the current one. For instance, generation 43 was created from generation 42 when we did ``` $ nix-env --install --attr nixpkgs.subversion nixpkgs.firefox ``` on a set of Nix expressions that contained Firefox and a new version of Subversion. Generations are grouped together into _profiles_ so that different users don’t interfere with each other if they don’t want to. For example: ``` $ ls -l /nix/var/nix/profiles/ ... lrwxrwxrwx 1 eelco ... default-42-link -> /nix/store/0c1p5z4kda11...-user-env lrwxrwxrwx 1 eelco ... default-43-link -> /nix/store/3aw2pdyx2jfc...-user-env lrwxrwxrwx 1 eelco ... default -> default-43-link ``` This shows a profile called `default`. The file `default` itself is actually a symlink that points to the current generation. When we do a `nix-env` operation, a new user environment and generation link are created based on the current one, and finally the `default` symlink is made to point at the new generation. This last step is atomic on Unix, which explains how we can do atomic upgrades. (Note that the building/installing of new packages doesn’t interfere in any way with old packages, since they are stored in different locations in the Nix store.) If you find that you want to undo a `nix-env` operation, you can just do ``` $ nix-env --rollback ``` which will just make the current generation link point at the previous link. E.g., `default` would be made to point at `default-42-link`. You can also switch to a specific generation: ``` $ nix-env --switch-generation 43 ``` which in this example would roll forward to generation 43 again. You can also see all available generations: ``` $ nix-env --list-generations ``` You generally wouldn’t have `/nix/var/nix/profiles/some-profile/bin` in your `PATH`. Rather, there is a symlink `~/.nix-profile` that points to your current profile. This means that you should put `~/.nix-profile/bin` in your `PATH` (and indeed, that’s what the initialisation script `/nix/etc/profile.d/nix.sh` does). This makes it easier to switch to a different profile. You can do that using the command `nix-env --switch-profile`: ``` $ nix-env --switch-profile /nix/var/nix/profiles/my-profile $ nix-env --switch-profile /nix/var/nix/profiles/default ``` These commands switch to the `my-profile` and default profile, respectively. If the profile doesn’t exist, it will be created automatically. You should be careful about storing a profile in another location than the `profiles` directory, since otherwise it might not be used as a root of the [garbage collector](garbage-collection.md). All `nix-env` operations work on the profile pointed to by `~/.nix-profile`, but you can override this using the `--profile` option (abbreviation `-p`): ``` $ nix-env --profile /nix/var/nix/profiles/other-profile --install --attr nixpkgs.subversion ``` This will _not_ change the `~/.nix-profile` symlink. ## [Sharing Packages Between Machines](https://docharvest.github.io/docs/nix/package-management/sharing-packages/) Contents nix Sharing Packages Between Machines Nix Sharing Packages Between Machines Sometimes you want to copy a package from one machine to another. Or, you want to install some packages and you know that another machine already has some or all of those packages or their dependencies. In that case there are mechanisms to quickly copy packages between machines. ## [Serving a Nix store via SSH](https://docharvest.github.io/docs/nix/package-management/ssh-substituter/) Contents nix Serving a Nix store via SSH Nix Serving a Nix store via SSH You can tell Nix to automatically fetch needed binaries from a remote Nix store via SSH. For example, the following installs Firefox, automatically fetching any store paths in Firefox’s closure if they are available on the server `avalon`: ``` $ nix-env --install --attr nixpkgs.firefox --substituters ssh://alice@avalon ``` This works similar to the binary cache substituter that Nix usually uses, only using SSH instead of HTTP: if a store path `P` is needed, Nix will first check if it’s available in the Nix store on `avalon`. If not, it will fall back to using the binary cache substituter, and then to building from source. > **Note** > > The SSH substituter currently does not allow you to enter an SSH passphrase interactively. Therefore, you should use `ssh-add` to load the decrypted private key into `ssh-agent`. You can also copy the closure of some store path, without installing it into your profile, e.g. ``` $ nix-store --realise /nix/store/m85bxg…-firefox-34.0.5 --substituters ssh://alice@avalon ``` This is essentially equivalent to doing ``` $ nix-copy-closure --from alice@avalon /nix/store/m85bxg…-firefox-34.0.5 ``` You can use SSH’s _forced command_ feature to set up a restricted user account for SSH substituter access, allowing read-only access to the local Nix store, but nothing more. For example, add the following lines to `sshd_config` to restrict the user `nix-ssh`: ``` Match User nix-ssh AllowAgentForwarding no AllowTcpForwarding no PermitTTY no PermitTunnel no X11Forwarding no ForceCommand nix-store --serve Match All ``` On NixOS, you can accomplish the same by adding the following to your `configuration.nix`: ``` nix.sshServe.enable = true; nix.sshServe.keys = [ "ssh-dss AAAAB3NzaC1k... bob@example.org" ]; ``` where the latter line lists the public keys of users that are allowed to connect. ## [Protocols](https://docharvest.github.io/docs/nix/protocols/) Contents nix Protocols Nix Protocols This chapter documents various developer-facing interfaces provided by Nix. ## [Binary Cache](https://docharvest.github.io/docs/nix/protocols/binary-cache/) Contents nix Binary Cache Nix Binary Cache The binary cache format is an interface designed for exposing a store over HTTP. A binary cache consists of: - A [`nix-cache-info`](./nix-cache-info.md) file at the root with remote-side configuration. - For each [store object](@docroot@/store/store-object.md): - A [`.narinfo`](./narinfo.md) file containing the object's [metadata](@docroot@/store/store-object.md#metadata) and a (usually relative) URL to the corresponding compressed NAR. - A possibly-compressed [Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) containing the store object's file system data. - For every entry in the [build trace](@docroot@/store/build-trace.md), a JSON file at `build-trace-v2//.doi`: - the path encodes the [key](@docroot@/protocols/json/build-trace-entry.md#key) - the contents are the [value](@docroot@/protocols/json/build-trace-entry.md#value). The following [store types](@docroot@/store/types/index.md) use the binary cache format: - [HTTP Binary Cache Store](@docroot@/store/types/http-binary-cache-store.md) — served over HTTP(S) - [Local Binary Cache Store](@docroot@/store/types/local-binary-cache-store.md) — stored on the file system - [S3 Binary Cache Store](@docroot@/store/types/s3-binary-cache-store.md) — stored in an AWS S3 bucket ## [.narinfo Format](https://docharvest.github.io/docs/nix/protocols/binary-cache/narinfo/) Contents nix \`.narinfo\` Format Nix \`.narinfo\` Format A `.narinfo` file contains the [metadata of a store object](@docroot@/store/store-object.md#metadata) in the [binary cache](@docroot@/protocols/binary-cache/index.md) format. It is a simple line-oriented format where each line is a `Key: Value` pair. Some keys (e.g. `Sig`) may appear multiple times. The file is named `.narinfo`, where `` is the [hash part](@docroot@/store/store-path.md#digest) of the store object's [store path](@docroot@/store/store-path.md). The fields correspond to those documented in the [store object info](@docroot@/protocols/json/store-object-info.md) JSON format: `.narinfo` field JSON field Differences `StorePath` [`path`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_path) Full [store path](@docroot@/store/store-path.md) rather than [store path base name](@docroot@/store/store-path.md#base-name) `URL` [`url`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_url) `Compression` [`compression`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_compression) Defaults to `bzip2` if omitted `FileHash` [`downloadHash`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_downloadHash) String-encoded hash rather than structured `FileSize` [`downloadSize`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_downloadSize) `NarHash` [`narHash`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_narHash) String-encoded hash rather than structured `NarSize` [`narSize`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_narSize) `References` [`references`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_references) Space-separated [store path base names](@docroot@/store/store-path.md#base-name) rather than a JSON array `Deriver` [`deriver`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_deriver) [Store path base name](@docroot@/store/store-path.md#base-name); `unknown-deriver` instead of `null` `Sig` [`signatures`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_signatures) May appear multiple times rather than using an array `CA` [`ca`](@docroot@/protocols/json/store-object-info.md#oneOf_i2_ca) String-encoded [content address](@docroot@/store/store-object/content-address.md) rather than structured ## Example ``` StorePath: /nix/store/n5wkd9frr45pa74if5gpz9j7mifg27fh-foo URL: nar/1w1fff338fvdw53sqgamddn1b2xgds473pv6y13gizdbqjv4i5p3.nar.xz?sha256=1w1fff338fvdw53sqgamddn1b2xgds473pv6y13gizdbqjv4i5p3 Compression: xz FileHash: sha256:09ymwqf5i9q7d4dm7x4pjjcqqj0qrcp5lnznbh42gfsci5hcbqqm FileSize: 4029176 NarHash: sha256:09ymwqf5i9q7d4dm7x4pjjcqqj0qrcp5lnznbh42gfsci5hcbqqm NarSize: 34878 References: g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-bar n5wkd9frr45pa74if5gpz9j7mifg27fh-foo Deriver: g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-bar.drv Sig: asdf:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== Sig: qwer:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== CA: fixed:r:sha256:1lr187v6dck1rjh2j6svpikcfz53wyl3qrlcbb405zlh13x0khhh ``` ## [nix-cache-info Format](https://docharvest.github.io/docs/nix/protocols/binary-cache/nix-cache-info/) Contents nix \`nix-cache-info\` Format Nix \`nix-cache-info\` Format The `nix-cache-info` file is a metadata file at the root of a [binary cache](@docroot@/protocols/binary-cache/index.md) (e.g., `https://cache.example.com/nix-cache-info`). MIME type: `text/x-nix-cache-info` ## Format Line-based key-value format: ``` Key: value ``` Leading and trailing whitespace is trimmed from values. Lines without a colon are ignored. Unknown keys are silently ignored. ## Fields ### `StoreDir` The Nix store directory path that this cache was built for (e.g., `/nix/store`). If present, Nix verifies that this matches the client's store directory: ``` error: binary cache 'https://example.com' is for Nix stores with prefix '/nix/store', not '/home/user/nix/store' ``` ### `WantMassQuery` `1` or `0`. Sets the default for [`want-mass-query`](@docroot@/store/types/http-binary-cache-store.md#store-http-binary-cache-store-want-mass-query). ### `Priority` Integer. Sets the default for [`priority`](@docroot@/store/types/http-binary-cache-store.md#store-http-binary-cache-store-priority). ## Example ``` StoreDir: /nix/store WantMassQuery: 1 Priority: 30 ``` ## Caching Behavior Nix caches `nix-cache-info` in the [cache directory](@docroot@/command-ref/env-common.md#env-NIX_CACHE_HOME) with a 7-day TTL. ## See Also - [HTTP Binary Cache Store](@docroot@/store/types/http-binary-cache-store.md) - [Serving a Nix store via HTTP](@docroot@/package-management/binary-cache-substituter.md) - [`substituters`](@docroot@/command-ref/conf-file.md#conf-substituters) ## [Derivation "ATerm" file format](https://docharvest.github.io/docs/nix/protocols/derivation-aterm/) Contents nix Derivation "ATerm" file format Nix Derivation "ATerm" file format For historical reasons, [store derivations](@docroot@/glossary.md#gloss-store-derivation) are stored on-disk in "Annotated Term" (ATerm) format ([guide](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html), [paper](https://doi.org/10.1002/\(SICI\)1097-024X\(200003\)30:3%3C259::AID-SPE298%3E3.0.CO;2-Y)). ## The ATerm format used Derivations are serialised in one of the following formats: - ``` Derive(...) ``` For all stable derivations. - ``` DrvWithVersion(, ...) ``` The only `version-string`s that are in use today are for [experimental features](@docroot@/development/experimental-features.md): - `"xp-dyn-drv"` for the [`dynamic-derivations`](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) experimental feature. ## Use for encoding to store object When derivation is encoded to a [store object](@docroot@/glossary.md#gloss-store-object) we make the following choices: - The store path [name](@docroot@/store/store-path.md#name) is the derivation name with `.drv` suffixed at the end Indeed, the ATerm format above does _not_ contain the name of the derivation, on the assumption that a store path will also be provided out-of-band. - The derivation is content-addressed using the ["Text" method](@docroot@/store/store-object/content-address.md#method-text) of content-addressing derivations Currently we always encode derivations to store object using the ATerm format (and the previous two choices), but we reserve the option to encode new sorts of derivations differently in the future. ## [JSON Formats](https://docharvest.github.io/docs/nix/protocols/json/) Contents nix JSON Formats Nix JSON Formats ## [Build Result](https://docharvest.github.io/docs/nix/protocols/json/build-result/) Contents nix Build Result Nix Build Result {{#include build-result-v1-fixed.md}} ## Examples ### Successful build ``` {{#include schema/build-result-v1/success.json}} ``` ### Failed build (output rejected) ``` {{#include schema/build-result-v1/output-rejected.json}} ``` ### Failed build (non-deterministic) ``` {{#include schema/build-result-v1/not-deterministic.json}} ``` ## [Build Trace Entry](https://docharvest.github.io/docs/nix/protocols/json/build-trace-entry/) Contents nix Build Trace Entry Nix Build Trace Entry {{#include build-trace-entry-v3-fixed.md}} ## Examples ### Simple build trace entry ``` {{#include schema/build-trace-entry-v3/simple.json}} ``` ### Build trace entry with signature ``` {{#include schema/build-trace-entry-v3/with-signature.json}} ``` ## [Content Address](https://docharvest.github.io/docs/nix/protocols/json/content-address/) Contents nix Content Address Nix Content Address {{#include content-address-v1-fixed.md}} ## Examples ### [Text](@docroot@/store/store-object/content-address.html#method-text) method ``` {{#include schema/content-address-v1/text.json}} ``` ### [Nix Archive](@docroot@/store/store-object/content-address.html#method-nix-archive) method ``` {{#include schema/content-address-v1/nar.json}} ``` ## [Derivation](https://docharvest.github.io/docs/nix/protocols/json/derivation/) Contents nix Derivation Nix Derivation {{#include ../derivation-v4-fixed.md}} ## [Options](https://docharvest.github.io/docs/nix/protocols/json/derivation/options/) Contents nix Options Nix Options {{#include ../derivation-options-v1-fixed.md}} ## Examples ### Input-addressed derivations #### Default options ``` {{#include ../schema/derivation-options-v1/ia/derivation-options/defaults.json}} ``` #### All options set ``` {{#include ../schema/derivation-options-v1/ia/derivation-options/all_set.json}} ``` #### Default options (structured attributes) ``` {{#include ../schema/derivation-options-v1/ia/derivation-options/structuredAttrs_defaults.json}} ``` #### All options set (structured attributes) ``` {{#include ../schema/derivation-options-v1/ia/derivation-options/structuredAttrs_all_set.json}} ``` ### Content-addressed derivations #### All options set ``` {{#include ../schema/derivation-options-v1/ca/derivation-options/all_set.json}} ``` #### All options set (structured attributes) ``` {{#include ../schema/derivation-options-v1/ca/derivation-options/structuredAttrs_all_set.json}} ``` ## [Resolved](https://docharvest.github.io/docs/nix/protocols/json/derivation/resolved/) Contents nix Resolved Nix Resolved {{#include ../derivation-resolved-v4-fixed.md}} ## [Deriving Path](https://docharvest.github.io/docs/nix/protocols/json/deriving-path/) Contents nix Deriving Path Nix Deriving Path {{#include deriving-path-v1-fixed.md}} ## Examples ### Constant ``` {{#include schema/deriving-path-v1/single_opaque.json}} ``` ### Output of static derivation ``` {{#include schema/deriving-path-v1/single_built.json}} ``` ### Output of dynamic derivation ``` {{#include schema/deriving-path-v1/single_built_built.json}} ``` ## [File System Object](https://docharvest.github.io/docs/nix/protocols/json/file-system-object/) Contents nix File System Object Nix File System Object {{#include file-system-object-v1-fixed.md}} ## Examples ### Simple ``` {{#include schema/file-system-object-v1/simple.json}} ``` ### Complex ``` {{#include schema/file-system-object-v1/complex.json}} ``` ## [Hash](https://docharvest.github.io/docs/nix/protocols/json/hash/) Contents nix Hash Nix Hash {{#include hash-v1-fixed.md}} ## Examples ### SHA-256 ``` {{#include schema/hash-v1/sha256.json}} ``` ### BLAKE3 ``` {{#include schema/hash-v1/blake3.json}} ``` ## [Signature](https://docharvest.github.io/docs/nix/protocols/json/signature/) Contents nix Signature Nix Signature {{#include signature-v2-fixed.md}} ## Examples ### Simple signature ``` {{#include schema/signature-v2/simple.json}} ``` ## [Store](https://docharvest.github.io/docs/nix/protocols/json/store/) Contents nix Store Nix Store {{#include store-v1-fixed.md}} ## Examples ### Empty store ``` {{#include schema/store-v1/empty.json}} ``` ### Store with one file ``` {{#include schema/store-v1/one-flat-file.json}} ``` ### Store with one derivation ``` {{#include schema/store-v1/one-derivation.json}} ``` ## [Store Object Info](https://docharvest.github.io/docs/nix/protocols/json/store-object-info/) Contents nix Store Object Info Nix Store Object Info {{#include store-object-info-v3-fixed.md}} ## Examples ### Minimal store object (content-addressed) ``` {{#include schema/store-object-info-v3/pure.json}} ``` ### Store object with impure fields ``` {{#include schema/store-object-info-v3/impure.json}} ``` ### Minimal store object (empty) ``` {{#include schema/store-object-info-v3/empty_pure.json}} ``` ### Store object with all impure fields ``` {{#include schema/store-object-info-v3/empty_impure.json}} ``` ### NAR info (minimal) ``` {{#include schema/nar-info-v3/pure.json}} ``` ### NAR info (with binary cache fields) ``` {{#include schema/nar-info-v3/impure.json}} ``` ## [Store Path](https://docharvest.github.io/docs/nix/protocols/json/store-path/) Contents nix Store Path Nix Store Path {{#include store-path-v1-fixed.md}} ## Examples ### Simple store path ``` {{#include schema/store-path-v1/simple.json}} ``` ## [Nix Archive (NAR) format](https://docharvest.github.io/docs/nix/protocols/nix-archive/) Contents nix Nix Archive (NAR) format Nix Nix Archive (NAR) format This is the complete specification of the [Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) format. The Nix Archive format closely follows the abstract specification of a [file system object](@docroot@/store/file-system-object.md) tree, because it is designed to serialize exactly that data structure. The format of this specification is close to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), with the exception of the `str(..)` function / parameterized rule, which length-prefixes and pads strings. This makes the resulting binary format easier to parse. Regular users do _not_ need to know this information. But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful. ``` nar = str("nix-archive-1"), nar-obj; nar-obj = str("("), nar-obj-inner, str(")"); nar-obj-inner = str("type"), str("regular") regular | str("type"), str("symlink") symlink | str("type"), str("directory") directory ; regular = [ str("executable"), str("") ], str("contents"), str(contents); symlink = str("target"), str(target); (* side condition: directory entries must be ordered by their names *) directory = { directory-entry }; directory-entry = str("entry"), str("("), str("name"), str(name), str("node"), nar-obj, str(")"); ``` The `str` function / parameterized rule is defined as follows: - `str(s)` = `int(|s|), pad(s);` - `int(n)` = the 64-bit little endian representation of the number `n` - `pad(s)` = the byte sequence `s`, padded with 0s to a multiple of 8 byte ## Kaitai Struct Specification The Nix Archive (NAR) format is also formally described using [Kaitai Struct](https://kaitai.io/), an Interface Description Language (IDL) for defining binary data structures. > Kaitai Struct provides a language-agnostic, machine-readable specification that can be compiled into parsers for various programming languages (e.g., C++, Python, Java, Rust). ``` {{#include nar.ksy}} ``` The source of the spec can be found [here](https://github.com/nixos/nix/blob/master/src/nix-manual/source/protocols/nix-archive/nar.ksy). Contributions and improvements to the spec are welcomed. ## [Nix32 Encoding](https://docharvest.github.io/docs/nix/protocols/nix32/) Contents nix Nix32 Encoding Nix Nix32 Encoding Nix32 is Nix's variant of [Base32](https://en.wikipedia.org/wiki/Base32) encoding, used for [store path digests](@docroot@/protocols/store-path.md), hash output via [`nix hash`](@docroot@/command-ref/new-cli/nix3-hash.md), and the [`outputHash`](@docroot@/language/advanced-attributes.md#adv-attr-outputHash) derivation attribute. ## Alphabet The Nix32 alphabet consists of these 32 characters: ``` 0 1 2 3 4 5 6 7 8 9 a b c d f g h i j k l m n p q r s v w x y z ``` The letters `e`, `o`, `u`, and `t` are omitted. ## Byte Order Nix32 encoding processes the hash bytes from the end (last byte first), while base-16 encoding processes from the beginning (first byte first). Consequently, the string sort order is determined primarily by the first bytes for base-16, and by the last bytes for Nix32. ## [Complete Store Path Calculation](https://docharvest.github.io/docs/nix/protocols/store-path/) Contents nix Complete Store Path Calculation Nix Complete Store Path Calculation This is the complete specification for how [store path](@docroot@/store/store-path.md)s are calculated. The format of this specification is close to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), but must deviate for a few things such as hash functions which we treat as bidirectional for specification purposes. Regular users do _not_ need to know this information --- store paths can be treated as black boxes computed from the properties of the store objects they refer to. But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful. ## Store path proper ``` store-path = store-dir "/" digest "-" name ``` where - `name` = the name of the store object. - `store-dir` = the [path of the store directory](@docroot@/store/store-path.md#store-directory-path) - `digest` = [Nix32](@docroot@/protocols/nix32.md) representation of the compressed to 160 bits [SHA-256](https://en.m.wikipedia.org/wiki/SHA-256) hash of `fingerprint`. For the definition of the hash compression algorithm, please refer to section 5.1 of the [Nix thesis](https://edolstra.github.io/pubs/phd-thesis.pdf). ## Fingerprint - ``` fingerprint = type ":sha256:" inner-digest ":" store ":" name ``` Note that it includes the location of the store as well as the name to make sure that changes to either of those are reflected in the hash (e.g. you won't get `/nix/store/-name1` and `/nix/store/-name2`, or `/gnu/store/-name1`, with equal hash parts). - `type` = one of: - ``` | "text" { ":" store-path } ``` This is for the ["Text"](@docroot@/store/store-object/content-address.md#method-text) method of content addressing store objects. The optional trailing store paths are the references of the store object. - ``` | "source" { ":" store-path } [ ":self" ] ``` This is for the ["Nix Archive"](@docroot@/store/store-object/content-address.md#method-nix-archive) method of content addressing store objects, if the hash algorithm is [SHA-256](https://en.m.wikipedia.org/wiki/SHA-256). Just like in the "Text" case, we can have the store objects referenced by their paths. Additionally, we can have an optional `:self` label to denote self-reference. - ``` | "output:" id ``` For either the outputs built from derivations, or content-addressed store objects that are not using one of the two above cases. To be explicit about the latter, that is currently these methods: - ["Flat"](@docroot@/store/store-object/content-address.md#method-flat) - ["Git"](@docroot@/store/store-object/content-address.md#method-git) - ["Nix Archive"](@docroot@/store/store-object/content-address.md#method-nix-archive) if the hash algorithm is not [SHA-256](https://en.m.wikipedia.org/wiki/SHA-256). `id` is the name of the output (usually, "out"). For content-addressed store objects, `id`, is always "out". - `inner-digest` = base-16 representation of a SHA-256 hash of `inner-fingerprint`. The base-16 encoding uses lower-cased hex digits. ## Inner fingerprint - `inner-fingerprint` = one of the following based on `type`: - if `type` = `"text:" ...`: the string written to the resulting store path. - if `type` = `"source:" ...`: the [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object. - if `type` = `"output:" id`: - For input-addressed derivation outputs: the [ATerm](@docroot@/protocols/derivation-aterm.md) serialization of the derivation modulo fixed output derivations. - For content-addressed store paths: ``` "fixed:out:" rec algo ":" hash ":" ``` where - `rec` = one of: - ``` | "" ``` (empty string) for hashes of the flat (single file) serialization - ``` | "r:" ``` hashes of the for [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) (arbitrary file system object) serialization - ``` | "git:" ``` hashes of the [Git blob/tree](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) [Merkel tree](https://en.wikipedia.org/wiki/Merkle_tree) format - ``` algo = "md5" | "sha1" | "sha256" ``` - `hash` = base-16 representation of the path or flat hash of the contents of the path (or expected contents of the path for fixed-output derivations). Note that `id` = `"out"`, regardless of the name part of the store path. Also note that NAR + SHA-256 must not use this case, and instead must use the `type` = `"source:" ...` case. ### Historical Note The `type` = `"source:" ...` and `type` = `"output:out"` grammars technically overlap in purpose, in that both can represent data hashed by its SHA-256 NAR serialization. The original reason for this way of computing names was to prevent name collisions (for security). For instance, the thinking was that it shouldn't be feasible to come up with a derivation whose output path collides with the path for a copied source. The former would have an `inner-fingerprint` starting with `output:out:`, while the latter would have an `inner-fingerprint` starting with `source:`. Since `64519cfd657d024ae6e2bb74cb21ad21b886fd2a` (2008), however, it was decided that separating derivation-produced vs manually-hashed content-addressed data like this was not useful. Now, data that is content-addressed with SHA-256 + NAR-serialization always uses the `source:...` construction, regardless of how it was produced (manually or by derivation). This allows freely switching between using [fixed-output derivations](@docroot@/glossary.md#gloss-fixed-output-derivation) for fetching, and fetching out-of-band and then manually adding. It also removes the ambiguity from the grammar. ## [Lockable HTTP Tarball Protocol](https://docharvest.github.io/docs/nix/protocols/tarball-fetcher/) Contents nix Lockable HTTP Tarball Protocol Nix Lockable HTTP Tarball Protocol Tarball flakes can be served as regular tarballs via HTTP or the file system (for `file://` URLs). Unless the server implements the Lockable HTTP Tarball protocol, it is the responsibility of the user to make sure that the URL always produces the same tarball contents. An HTTP server can return an "immutable" HTTP URL appropriate for lock files. This allows users to specify a tarball flake input in `flake.nix` that requests the latest version of a flake (e.g. `https://example.org/hello/latest.tar.gz`), while `flake.lock` will record a URL whose contents will not change (e.g. `https://example.org/hello/.tar.gz`). To do so, the server must return an [HTTP `Link` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Link) with the `rel` attribute set to `immutable`, as follows: ``` Link: ; rel="immutable" ``` (Note the required `<` and `>` characters around _flakeref_.) _flakeref_ must be a tarball flakeref. It can contain the tarball flake attributes `narHash`, `rev`, `revCount` and `lastModified`. If `narHash` is included, its value must be the \[NAR hash\]\[Nix Archive\] of the unpacked tarball (as computed via `nix hash path`). Nix checks the contents of the returned tarball against the `narHash` attribute. The `rev` and `revCount` attributes are useful when the tarball flake is a mirror of a fetcher type that has those attributes, such as Git or GitHub. They are not checked by Nix. ``` Link: ; rel="immutable" ``` (The linebreaks in this example are for clarity and must not be included in the actual response.) For tarball flakes, the value of the `lastModified` flake attribute is defined as the timestamp of the newest file inside the tarball. ## Gitea and Forgejo support This protocol is supported by Gitea since v1.22.1 and by Forgejo since v7.0.4/v8.0.0 and can be used with the following flake URL schema: ``` https://///archive/.tar.gz ``` > **Example** > > ``` > # flake.nix > { > inputs = { > foo.url = "https://gitea.example.org/some-person/some-flake/archive/main.tar.gz"; > bar.url = "https://gitea.example.org/some-other-person/other-flake/archive/442793d9ec0584f6a6e82fa253850c8085bb150a.tar.gz"; > qux = { > url = "https://forgejo.example.org/another-person/some-non-flake-repo/archive/development.tar.gz"; > flake = false; > }; > }; > outputs = { foo, bar, qux }: { /* ... */ }; > } > ``` ``` [Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive ``` ## [Quick Start](https://docharvest.github.io/docs/nix/quick-start/) Contents nix Quick Start Nix Quick Start This chapter is for impatient people who don't like reading documentation. For more in-depth information you are kindly referred to subsequent chapters. 1. Install Nix: ``` $ curl -L https://nixos.org/nix/install | sh ``` The install script will use `sudo`, so make sure you have sufficient rights. For other installation methods, see the detailed [installation instructions](installation/index.md). 2. Run software without installing it permanently: ``` $ nix-shell --packages cowsay lolcat ``` This downloads the specified packages with all their dependencies, and drops you into a Bash shell where the commands provided by those packages are present. This will not affect your normal environment: ``` [nix-shell:~]$ cowsay Hello, Nix! | lolcat ``` Exiting the shell will make the programs disappear again: ``` [nix-shell:~]$ exit $ lolcat lolcat: command not found ``` 3. Search for more packages on [search.nixos.org](https://search.nixos.org/) to try them out. 4. Free up storage space: ``` $ nix-collect-garbage ``` ## [Nix Release Notes](https://docharvest.github.io/docs/nix/release-notes/) Contents nix Nix Release Notes Nix Nix Release Notes The Nix release cycle is calendar-based as follows: Nix has a release cycle of roughly 6 weeks. Notable changes and additions are announced in the release notes for each version. The supported Nix versions are: - The latest release - The version used in the stable NixOS release, which is announced in the [NixOS release notes](https://nixos.org/manual/nixos/stable/release-notes.html#ch-release-notes). Bugfixes and security issues are backported to every supported version. Patch releases are published as needed. ## [Nix Store](https://docharvest.github.io/docs/nix/store/) Contents nix Nix Store Nix Nix Store The _Nix store_ is an abstraction to store immutable file system data (such as software packages) that can have dependencies on other such data. Concretely, albeit using concepts that are only defined in the rest of the chapter, a store consists of: - A set of [store objects](./store-object.md), the immutable file system data. This can also be looked at as a map from [store paths](./store-path.md) to store objects. - A set of [derivations](./derivation/index.md), instructions for building store objects. This can also be looked at as a map from [store paths](./store-path.md) to derivations. Since store paths to derivations always end in `.drv`, and store paths to other store objects never do, the two maps can also be combined into one. Derivations can also be encoded as store objects too. - A [build trace](./build-trace.md), a record of which derivations have been built and what they produced. > **Warning** > > The concept of a build trace is currently [**experimental**](@docroot@/development/experimental-features.md#xp-feature-ca-derivations) and subject to change. There are [multiple types of Nix stores](./types/index.md) with different capabilities, such as the default one on the [local file system](./types/local-store.md) (`/nix/store`) or [binary caches](./types/http-binary-cache-store.md). ## [Build Trace](https://docharvest.github.io/docs/nix/store/build-trace/) Contents nix Build Trace Nix Build Trace > **Warning** > > This entire concept is currently [**experimental**](@docroot@/development/experimental-features.md#xp-feature-ca-derivations) and subject to change. The _build trace_ is a [memoization table](https://en.wikipedia.org/wiki/Memoization) for builds. It maps the inputs of builds to the outputs of builds. Each _\[entry\]{#entry}_ in the build trace maps a [derivation](./derivation/index.md) to a map of [output](./derivation/outputs/index.md) names to [store objects](@docroot@/store/store-object.md). In general the derivations used as a key should be [_resolved_](./resolution.md). A build trace with all-resolved-derivation keys is also called a _base build trace_ for extra clarity. If all the resolved inputs of a derivation are content-addressed, that means the inputs will be fully determined, leaving no ambiguity for what build was performed. (Input-addressed inputs however are still ambiguous. They too should be locked down, but this is left as future work.) Accordingly, to look up an unresolved derivation, one must first resolve it to get a resolved derivation. Resolving itself involves looking up entries in the build trace, so this is a mutually recursive process that will end up inspecting possibly many entries. Except for the issue with input-addressed paths called out above, base build traces are trivially _coherent_ -- incoherence is not possible. That means that the claims that each key-value base build try entry makes are independent, and no mapping invalidates another mapping. Whether the mappings are _true_, i.e. the faithful recording of actual builds performed, is another matter. Coherence is about the multiple claims of the build trace being mutually consistent, not about whether the claims are individually true or false. In general, there is no way to audit a build trace entry except for by performing the build again from scratch. And even in that case, a different result doesn't mean the original entry was a "lie", because the derivation being built may be non-deterministic. As such, the decision of whether to trust a counterparty's build trace is a fundamentally subject policy choice. Build trace entries are typically _signed_ in order to enable arbitrary public-key-based trust polices. ## Derived build traces {#derived} Implementations that wish to memoize the above may also keep additional _derived_ build trace entries that do map unresolved derivations. But if they do so, they _must_ also keep the underlying base entries with resolved derivation keys around. Firstly, this ensures that the derived entries are merely cache, which could be recomputed from scratch. Secondly, this ensures the coherence of the derived build trace. Unlike with base build traces, incoherence with derived build traces is possible. The key ingredient is that derivation resolution is only deterministic with respect to a fixed base build trace. Without fixing the base build trace, it inherits the subjectivity of base build traces themselves. Concretely, suppose there are three derivations \\(a\\), \\(b\\), and \\(c\\). Let \\(a\\) be a resolved derivation, but let \\(b\\) and \\(c\\) be unresolved and both take as an input an output of \\(a\\). Now suppose that derived entries are made for \\(b\\) and \\(c\\) based on two different entries of \\(a\\). (This could happen if \\(a\\) is non-deterministic, \\(a\\) and \\(b\\) are built in one store, \\(a\\) and \\(c\\) are built in another store, and then a third store substitutes from both of the first two stores.) If trusting the derived build trace entries for \\(b\\) and \\(c\\) requires that each's underlying entry for \\(a\\) be also trusted, the two different mappings for \\(a\\) will be caught. However, if \\(b\\) and \\(c\\)'s entries can be combined in isolation, there will be nothing to catch the contradiction in their hidden assumptions about \\(a\\)'s output. ## [Building](https://docharvest.github.io/docs/nix/store/building/) Contents nix Building Nix Building As discussed in the [main page on derivations](./derivation/index.md): > A derivation is a specification for running an executable on precisely defined input to produce one or more [store objects](./store-object.md). This page describes _building_ a derivation, which is to say following the instructions in the derivation to actually run the executable. Some elements of derivations are self-explanatory. For example, the arguments specified in the derivation really are the arguments passed to the executable. In other cases, however, there is additional common steps performed by Nix for all derivations --- mostly for setting up the build environment and collecting the built outputs. The chief design consideration for the building process is _determinism_. Conventional operating systems are typically not designed with determinism in mind. But determinism is needed to make Nix's build caching a transparent abstraction. > **Explanation** > > For example, no one wants to slightly modify a derivation, and then find that it no longer builds for an unrelated reason, because the original derivation _also_ doesn't build anymore, but the cache hit on the original derivation was hiding this. We want builds that succeed once to continue succeeding, to encourage fearless modification of old build recipes. Determinism is what enables things that once worked to keep working. The life cycle of a build can be broken down into 3 parts: 1. Spawn the builder process with the proper environment, including the correct process arguments, environment variables, and file system state. 2. Wait for the builder process to exit and collect its exit status. Exit code 0 means success; anything else is a build failure. (Strictly speaking, Nix detects process exit by waiting for the standard output and error streams to close. If a builder explicitly closes these streams without exiting, Nix will kill it, and deem the build a failure. Processes should therefore exit _without_ explicitly closing those standard streams, and let the exiting of the process close them implicitly.) Nix also logs the standard output and error of the process, but this is just for human convenience and does not influence the behavior of the system. (Builder processes have no idea what the consumer of their standard output and error does with the pseudo-terminal master, only that they are indeed consumed so buffers do not fill up etc. and writes to each output standard stream will continue to succeed. In practice, Nix will store the log in `/nix/var/log/nix`) 3. Processing the outputs after the builder has exited. The builder process on exit should have left behind files for each output the derivation is supposed to produce. The files must be processed to turn them into bona fide store objects. If the processing succeeds, those store objects are associated with the derivation as (the results of) a successful build. Step (3) is done by Nix externally to the build itself, which is just steps (1) and (2). In step (3), just inert data is processed, since the builder process has exited or been killed by then. Step (1) however is best described not from Nix's perspective, but from the build process's perspective. > **Explanation** > > Ultimately, what matters for determinism is what the build process can observe: what resources (files, networking, etc.) it can see, what syscalls succeed or fail, etc. Nix can achieve this through many different sandboxing strategies (namespaces, VMs, chroots, ...), but the process shouldn't be able to tell them apart. We therefore specify building from the process's perspective, not Nix's perspective, to focus on _what_, not _how_. ## What derivations can be built Actually only some derivations are ready to be built. In particular, only [_resolved_](./resolution.md) derivations can be built. That is to say, a derivation that depends on other derivations is not ready yet to be built, because some of those other derivations might not have yet been built. If the other derivations are indeed all built, we can witness this fact by resolving the derivation, and converting all the derivation's input references into plain store paths. > **Note** > > Note that [input-addressing](derivation/outputs/input-address.md) derivations are improperly resolved. As discussed on the linked page, the current input-addressing algorithm does not respect resolution-equivalence of derivations (\\(\\sim\_\\mathrm{Drv}\\)). That means that if Nix properly resolved an input-addressed derivation, the resolved derivation would have different input addresses, violating expectations. Nix therefore improperly resolves the derivation, keeping its original input-addressed output paths, creating an invalid derivation that is both resolved and instructed to create the outputs at the originally expected paths. ## Environment of the builder process This section describes how the [`builder`](./derivation/index.md#builder) is executed. > **Implementation detail** > > Nix prevents multiple [Nix instances](@docroot@/glossary.md#gloss-nix-instance) from performing the same build at the same time, for example by acquiring exclusive file locks. ### File system The builder should have access to a limited file system where only certain objects are available. The most important exposed files are the inputs (other store objects) of the (resolved) derivation. Additionally, some other files are exposed. #### Store inputs The builder will be run against a file system in which the [store directory](./store-path.md#store-directory-path) contains the [closure](@docroot@/glossary.md#gloss-closure) of the inputs. In particular, consider a store that just contains this closure. That store is exposed to the file system according to the rules specified in the [Exposing Store Objects in OS File Systems](./store-path.md#exposing) documentation. This precisely defines the file system layout of the store that should be visible to the builder process. > **Note** > > Historically, Nix exposed _at least_ the following store contents to the builder, but also arbitrarily other store objects, due to limitations around operating systems' file system virtualization capabilities, and wanting to avoid copying or moving files. It still can do this in so-called _unsandboxed_ builds. > > Such builds should be considered discouraged, but one that works less badly against non-mischievous derivations than might be expected. This is because store paths are relatively unpredictable, so a well-behaved program is unlikely to stumble upon a store object it wasn't supposed to know about. > > As operating systems developed better file system primitives, the need for disabling sandboxing has lessened greatly over the years, and this trend should continue into the future. The outputs are expected to be created in that store directory as if they were valid store objects. (They are just files during builder execution, but during [processing outputs](#processing-outputs) they will be turned into proper store objects.) The [environment variables](#env-vars) for each output indicate where the builder should write them; Nix ensures that those paths do not yet exist when the builder is run. > **Note** > > In sandboxed builds, ensuring that the outputs do not exist in the store directory is trivial. In unsandboxed builds, it is harder in general. In the worst case, the derivation is in fact rewritten so different output paths are used instead, and then the outputs are rewritten back to the intended output paths after. In the content-addressing case rewriting would be needed either way, but in the input-addressing case, this is a significant degradation, as the point of input addressing is to avoid rewrites by knowing output paths in advance. ### Other file system state - The current working directory of the builder process will be a fresh temporary directory. It is initially empty when the process starts except for a few input files: - If [`__structuredAttrs`](@docroot@/language/advanced-attributes.md#adv-attr-structuredAttrs) is enabled: `.attrs.json` (the derivation attributes as JSON) and `.attrs.sh` (a Bash-compatible rendering of the same). The environment variables `NIX_ATTRS_JSON_FILE` and `NIX_ATTRS_SH_FILE` point to these files, respectively. - If [`passAsFile`](@docroot@/language/advanced-attributes.md#adv-attr-passAsFile) is used (only without `__structuredAttrs`): for each attribute name listed, a file `.attr-` where `` is the [Nix32](@docroot@/protocols/nix32.md)\-encoded SHA-256 hash of the attribute name. The environment variable `Path` points to the file containing the attribute's value. In sandboxed builds, this directory is at a deterministic path inside the sandbox (controlled by the [`sandbox-build-dir`](@docroot@/command-ref/conf-file.md#conf-sandbox-build-dir) setting, default `/build`). See also the per-store [`build-dir`](@docroot@/store/types/local-store.md#store-local-store-build-dir) setting for the host-side location. - Basic device nodes for essential operations (null device, random number generation, standard streams as a pseudo terminal) (A pseudo terminal would not be strictly necessary since the standard streams are passively logging, not there to facilitate interaction. But it is still useful to entice programs to do nicer logging with e.g. colors etc.) - On Linux: Process information via `/proc` - Minimal user and group identity information - A loopback-only network configuration with hostname set to `localhost` > **Note** > > Fixed-output derivations have access to additional operating system state to facilitate communication with the outside world, such as network name resolution and TLS certificate verification. This is necessary because these derivations are allowed to access the network, unlike regular derivations which are fully sandboxed. ### Environment variables {#env-vars} The environment is cleared and set to the derivation attributes, as specified above. For most derivations types this must contain at least: - For each output declared in `outputs`, the corresponding environment variable is set to point to the intended path in the Nix store for that output. Each output path is a concatenation of the cryptographic hash of all build inputs, the `name` attribute and the output name. (The output name is omitted if it's `out`.) In addition, the following variables are set: - `NIX_BUILD_TOP` contains the path of the temporary directory for this build. - Also, `TMPDIR`, `TEMPDIR`, `TMP`, `TEMP` are set to point to the temporary directory. This is to prevent the builder from accidentally writing temporary files anywhere else. Doing so might cause interference by other processes. - `PATH` is set to `/path-not-set` to prevent shells from initialising it to their built-in default value. - `HOME` is set to `/homeless-shelter`. (Without sandboxing, this discourages programs from using `/etc/passwd` or the like to find the user's home directory, which could cause impurity.) Usually, when `HOME` is set, it is used as the location of the home directory, even if it points to a non-existent path. - `NIX_STORE` is set to the path of the top-level Nix [store directory path](./store-path.md#store-directory-path) (typically, `/nix/store`). - `NIX_ATTRS_JSON_FILE` & `NIX_ATTRS_SH_FILE` if `__structuredAttrs` is set to `true` for the derivation. A detailed explanation of this behavior can be found in the [section about structured attrs](@docroot@/language/advanced-attributes.md#adv-attr-structuredAttrs). ### Arguments The builder is passed the arguments specified by the derivation attribute `args`. ## Processing outputs If the builder exited successfully, the following steps happen in order to turn the output directories left behind by the builder into proper store objects: - **Normalize the file permissions** The files must conform to the model described in the [Exposing in OS file systems](./file-system-object/os-file-system.md) section. For example, timestamps and permissions are canonicalised. - **Calculate the references** Nix scans each output path for [references](./store-object.md#references) to input store objects by looking for the [digest](./store-path.md#digest) of each input. (The name part and the [store directory path](./store-path.md#store-directory-path) are ignored when scanning; an input's hash part that is neither followed by a `-` nor proceeded by a `/` still scans as a reference.) Since these are potential runtime dependencies, Nix will register them as references of the output store object they occur in. Nix also scans for references from one output to another in the same way, because outputs are allowed to refer to each other. The outputs' references must form a [directed acyclic graph](@docroot@/glossary.md#gloss-directed-acyclic-graph). (This is not a special restriction for outputs; it is true for the references of all store objects in general.) In the case of derivations with output paths that are fixed in advance (i.e. [input-addressing](./derivation/outputs/input-address.md) derivations, or [fixed content-addressing](./derivation/outputs/content-address.md#fixed) derivations), the actual final store path to each output is used during the build if possible. For [floating content-addressing](./derivation/outputs/content-address.md#floating) derivations, however, the final store path is not known in advance by definition. Scratch store paths must therefore be used instead. Scanning will use those scratch paths, but then any output-to-be that contains such a scanned scratch path must be rewritten to instead use the final (content-addressed) path of the output in question. At this point, the file system data is in the proper form, and the valid acyclic reference data for each output is also calculated, so the outputs are added to the store as proper store objects. Additionally, those store objects (at least in the case that they are [content-addressed](./derivation/outputs/content-address.md)) can be associated with the derivation in the [build trace](./build-trace.md) in the record for a successful build. > **Implementation detail** > > Nix will normally clean up and remove the temporary build directory after every build, successful or unsuccessful. The builder doesn't know whether Nix does or not, however, as it will have exited before the build directory is cleaned up, and it will not see any old build directory if (after a failed build) it is run again. The [`--keep-failed`](@docroot@/command-ref/opt-common.md#opt-keep-failed) option can be specified to keep the build directory in the case of a failing build. ## [Store Derivation and Deriving Path](https://docharvest.github.io/docs/nix/store/derivation/) Contents nix Store Derivation and Deriving Path Nix Store Derivation and Deriving Path Besides functioning as a [content-addressed store](@docroot@/glossary.md#gloss-content-addressed-store), the Nix store layer works as a [build system](@docroot@/glossary.md#gloss-build-system). Other systems (like Git or IPFS) also store and transfer immutable data, but they don't concern themselves with _how_ that data was created. This is where Nix distinguishes itself. _Derivations_ represent individual build steps, and _deriving paths_ are needed to refer to the _outputs_ of those build steps before they are built. ## Store Derivation {#store-derivation} A derivation is a specification for running an executable on precisely defined input to produce one or more [store objects](@docroot@/store/store-object.md). These store objects are known as the derivation's _outputs_. Derivations are _built_, in which case the process is spawned according to the spec, and when it exits, required to leave behind files which will (after post-processing) become the outputs of the derivation. This process is described in detail in [Building](@docroot@/store/building.md). A derivation consists of: - A name - An [inputs specification](#inputs), a set of [deriving paths](#deriving-path) - An [outputs specification](./outputs/index.md), specifying which outputs should be produced, and various metadata about them. - The ["system" type](#system) (e.g. `x86_64-linux`) where the executable is to run. - The [process creation fields](#process-creation-fields): to spawn the arbitrary process which will perform the build step. ### Referencing derivations {#derivation-path} Derivations are always referred to by the [store path](@docroot@/store/store-path.md) of the store object they are encoded to. See the [encoding section](#derivation-encoding) for more details on how this encoding works, and thus what exactly what store path we would end up with for a given derivation. The store path of the store object which encodes a derivation is often called a _derivation path_ for brevity. ## Deriving path {#deriving-path} Deriving paths are a way to refer to [store objects](@docroot@/store/store-object.md) that may or may not yet be [realised](@docroot@/glossary.md#gloss-realise). There are two forms: - \[_constant_\]{#deriving-path-constant}: just a [store path](@docroot@/store/store-path.md). It can be made [valid](@docroot@/glossary.md#gloss-validity) by copying it into the store: from the evaluator, command line interface or another store. - \[_output_\]{#deriving-path-output}: a pair of a [store path](@docroot@/store/store-path.md) to a [store derivation](#store-derivation) and an [output](./outputs/index.md) name. In pseudo code: ``` type OutputName = String; type ConstantPath = { path: StorePath; }; type OutputPath = { drvPath: StorePath; output: OutputName; }; type DerivingPath = ConstantPath | OutputPath; ``` Deriving paths are necessary because, in general and particularly for [content-addressing derivations](@docroot@/glossary.md#gloss-content-addressing-derivation), the [store path](@docroot@/store/store-path.md) of an [output](./outputs/index.md) is not known in advance. We can use an output deriving path to refer to such an output, instead of the store path which we do not yet know. ## Parts of a derivation A derivation is constructed from the parts documented in the following subsections. ### Inputs {#inputs} The inputs are a set of [deriving paths](#deriving-path), referring to all store objects needed in order to perform this build step. The [process creation fields](#process-creation-fields) will presumably include many [store paths](@docroot@/store/store-path.md): - The path to the executable normally starts with a store path - The arguments and environment variables likely contain many other store paths. But rather than somehow scanning all the other fields for inputs, Nix requires that all inputs be explicitly collected in the inputs field. It is instead the responsibility of the creator of a derivation (e.g. the evaluator) to ensure that every store object referenced in another field (e.g. referenced by store path) is included in this inputs field. ### System {#system} The system type on which the [`builder`](#builder) executable is meant to be run. A necessary condition for Nix to schedule a given derivation on some [Nix instance](@docroot@/glossary.md#gloss-nix-instance) is for the "system" of that derivation to match that instance's [`system` configuration option](@docroot@/command-ref/conf-file.md#conf-system) or [`extra-platforms` configuration option](@docroot@/command-ref/conf-file.md#conf-extra-platforms). By putting the `system` in each derivation, Nix allows _heterogeneous_ build plans, where not all steps can be run on the same machine or same sort of machine. Nix can schedule builds such that it automatically builds on other platforms by [forwarding build requests](@docroot@/advanced-topics/distributed-builds.md) to other Nix instances. ### Process creation fields {#process-creation-fields} These are the three fields which describe how to spawn the process which (along with any of its own child processes) will perform the build. You may note that this has everything needed for an `execve` system call. #### Builder {#builder} This is the path to an executable that will perform the build and produce the [outputs](./outputs/index.md). #### Arguments {#args} Command-line arguments to be passed to the [`builder`](#builder) executable. Note that these are the arguments after the first argument. The first argument passed to the `builder` will be the value of `builder`, as per the usual convention on Unix. See [Wikipedia](https://en.wikipedia.org/wiki/Argv) for details. #### Environment Variables {#env} Environment variables which will be passed to the [builder](#builder) executable. #### Structured Attributes {#structured-attrs} Nix also has special support for embedding JSON in the derivations. The environment variable `NIX_ATTRS_JSON_FILE` points to the exact location of that file both in a build and a [`nix-shell`](@docroot@/command-ref/nix-shell.md). As a convenience to Bash builders, Nix writes a script that initialises shell variables corresponding to all attributes that are representable in Bash. The environment variable `NIX_ATTRS_SH_FILE` points to the exact location of the script, both in a build and a [`nix-shell`](@docroot@/command-ref/nix-shell.md). This includes non-nested (associative) arrays. For example, the attribute `hardening.format = true` ends up as the Bash associative array element `${hardening[format]}`. ### Placeholders Placeholders are opaque values used within the [process creation fields](#process-creation-fields) to \[store objects\] for which we don't yet know [store path](@docroot@/store/store-path.md)s. They are strings in the form `/` that are embedded anywhere within the strings of those fields, and we are [considering](https://github.com/NixOS/nix/issues/12361) to add store-path-like placeholders. > **Note** > > Output Deriving Path exist to solve the same problem as placeholders --- that is, referring to store objects for which we don't yet know a store path. They also have a string syntax with `^`, [described in the encoding section](#deriving-path-encoding). We could use that syntax instead of `/` for placeholders, but its human-legibility would cause problems. There are two types of placeholder, corresponding to the two cases where this problem arises: - \[Output placeholder\]{#output-placeholder}: This is a placeholder for a derivation's own output. - \[Input placeholder\]{#input-placeholder}: This is a placeholder to a derivation's non-constant [input](#inputs), i.e. an input that is an \[output derived path\]. > **Explanation** > > In general, we need to [realise](@docroot@/glossary.md#gloss-realise) a [store object](@docroot@/store/store-object.md) in order to be sure to have a store object for it. But for these two cases this is either impossible or impractical: > > - In the output case this is impossible: > > We cannot build the output until we have a correct derivation, and we cannot have a correct derivation (without using placeholders) until we have the output path. > > - In the input case this is impractical: > > If we always build a dependency first, and then refer to its output by store path, we would lose the ability for a derivation graph to describe an entire build plan consisting of multiple build steps. > ## Encoding ### Derivation {#derivation-encoding} There are two formats, documented separately: - The legacy ["ATerm" format](@docroot@/protocols/derivation-aterm.md) - The experimental, currently under development and changing [JSON format](@docroot@/protocols/json/derivation/index.md) Every derivation has a canonical choice of encoding used to serialize it to a store object. This ensures that there is a canonical [store path](@docroot@/store/store-path.md) used to refer to the derivation, as described in [Referencing derivations](#derivation-path). > **Note** > > Currently, the canonical encoding for every derivation is the "ATerm" format, but this is subject to change for the types of derivations which are not yet stable. Regardless of the format used, when serializing a derivation to a store object, that store object will be content-addressed. In the common case, the inputs to store objects are either: - [constant deriving paths](#deriving-path-constant) for content-addressed source objects, which are "initial inputs" rather than the outputs of some other derivation - the outputs of other derivations If those other derivations _also_ abide by this common case (and likewise for transitive inputs), then the entire closure of the serialized derivation will be content-addressed. ### Deriving Path {#deriving-path-encoding} - _constant_ Constant deriving paths are encoded simply as the underlying store path is. Thus, we see that every encoded store path is also a valid encoded (constant) deriving path. - _output_ Output deriving paths are encoded by - encoding of a store path referring to a derivation - a `^` separator (or `!` in some legacy contexts) - the name of an output of the previously referred derivation > **Example** > > ``` > /nix/store/lxrn8v5aamkikg6agxwdqd1jz7746wz4-firefox-98.0.2.drv^out > ``` > > This parses like so: > > ``` > /nix/store/lxrn8v5aamkikg6agxwdqd1jz7746wz4-firefox-98.0.2.drv^out > |------------------------------------------------------------| |-| > store path (usual encoding) output name > |--| > note the ".drv" > ``` ## Extending the model to be higher-order {#dynamic} **Experimental feature**: [`dynamic-derivations`](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) So far, we have used store paths to refer to derivations. That works because we've implicitly assumed that all derivations are created _statically_ --- created by some mechanism out of band, and then manually inserted into the store. But what if derivations could also be created dynamically within Nix? In other words, what if derivations could be the outputs of other derivations? > **Note** > > In the parlance of "Build Systems à la carte", we are generalizing the Nix store layer to be a "Monadic" instead of "Applicative" build system. How should we refer to such derivations? A deriving path works, the same as how we refer to other derivation outputs. But what about a dynamic derivations output? (i.e. how do we refer to the output of a derivation, which is itself an output of a derivation?) For that we need to generalize the definition of deriving path, replacing the store path used to refer to the derivation with a nested deriving path: ``` type OutputPath = { - drvPath: StorePath; + drvPath: DerivingPath; output: OutputName; }; ``` Now, the `drvPath` field of `OutputPath` is itself a `DerivingPath` instead of a `StorePath`. With that change, here is updated definition: ``` type OutputName = String; type ConstantPath = { path: StorePath; }; type OutputPath = { drvPath: DerivingPath; output: OutputName; }; type DerivingPath = ConstantPath | OutputPath; ``` Under this extended model, `DerivingPath`s are thus inductively built up from a root `ConstantPath`, wrapped with zero or more outer `OutputPath`s. ### Encoding {#deriving-path-encoding-higher-order} The encoding is adjusted in the natural way, encoding the `drv` field recursively using the same deriving path encoding. The result of this is that it is possible to have a chain of `^` at the end of the final string, as opposed to just a single one. > **Example** > > ``` > /nix/store/lxrn8v5aamkikg6agxwdqd1jz7746wz4-firefox-98.0.2.drv^foo.drv^bar.drv^out > |----------------------------------------------------------------------------| |-| > inner deriving path (usual encoding) output name > |--------------------------------------------------------------------| |-----| > even more inner deriving path (usual encoding) output name > |------------------------------------------------------------| |-----| > innermost constant store path (usual encoding) output name > ``` ## [Derivation Outputs and Types of Derivations](https://docharvest.github.io/docs/nix/store/derivation/outputs/) Contents nix Derivation Outputs and Types of Derivations Nix Derivation Outputs and Types of Derivations As stated on the [main pages on derivations](../index.md#store-derivation), a derivation produces [store objects](@docroot@/store/store-object.md), which are known as the _outputs_ of the derivation. Indeed, the entire point of derivations is to produce these outputs, and to reliably and reproducibly produce these derivations each time the derivation is run. One of the parts of a derivation is its _outputs specification_, which specifies certain information about the outputs the derivation produces when run. The outputs specification is a map, from names to specifications for individual outputs. ## Output Names {#outputs} Output names can be any string which is also a valid [store path name](@docroot@/store/store-path.md#name). The name mapped to each output specification is not actually the name of the output. In the general case, the output store object has name `derivationName + "-" + outputSpecName`, not any other metadata about it. However, an output spec named "out" describes and output store object whose name is just the derivation name. > **Example** > > A derivation is named `hello`, and has two outputs, `out`, and `dev` > > - The derivation's path will be: `/nix/store/-hello.drv`. > > - The store path of `out` will be: `/nix/store/-hello`. > > - The store path of `dev` will be: `/nix/store/-hello-dev`. > The outputs are the derivations are the [store objects](@docroot@/store/store-object.md) it is obligated to produce. > **Note** > > The formal terminology here is somewhat at odds with everyday communication in the Nix community today. "output" in casual usage tends to refer to either to the actual output store object, or the notional output spec, depending on context. > > For example "hello's `dev` output" means the store object referred to by the store path `/nix/store/-hello-dev`. It is unusual to call this the "`hello-dev` output", even though `hello-dev` is the actual name of that store object. ## Types of output addressing The main information contained in an output specification is how the derivation output is addressed. In particular, the specification decides: - whether the output is [content-addressed](./content-address.md) or [input-addressed](./input-address.md) - if the content is content-addressed, how is it content addressed - if the content is content-addressed, [what is its content address](./content-address.md#fixed) (and thus what is its \[store path\]) ## Types of derivations The sections on each type of derivation output addressing ended up discussing other attributes of the derivation besides its outputs, such as purity, scheduling, determinism, etc. This is no concidence; for the type of a derivation is in fact one-for-one with the type of its outputs: - A derivation that produces _xyz-addressed_ outputs is an _xyz-addressing_ derivations. The rules for this are fairly concise: - All the outputs must be of the same type / use the same addressing - The derivation must have at least one output - Additionally, if the outputs are fixed content-addressed, there must be exactly one output, whose specification is mapped from the name `out`. (The name `out` is special, according to the rules described above. Having only one output and calling its specification `out` means the single output is effectively anonymous; the store path just has the derivation name.) (This is an arbitrary restriction that could be lifted.) - The output is either _fixed_ or _floating_, indicating whether the store path is known prior to building it. - With fixed content-addressing it is fixed. > A _fixed content-addressing_ derivation is also called a _fixed-output derivation_, since that is the only currently-implemented form of fixed-output addressing - With floating content-addressing or input-addressing it is floating. > Thus, historically with Nix, with no experimental features enabled, _all_ outputs are fixed. - The derivation may be _pure_ or _impure_, indicating what read access to the outside world the [builder](../index.md#builder) has. - An input-addressing derivation _must_ be pure. > If it is impure, we would have a large problem, because an input-addressed derivation always produces outputs with the same paths. - A content-addressing derivation may be pure or impure - If it is impure, it may be fixed (typical), or it may be floating if the additional [`impure-derivations`](@docroot@/development/experimental-features.md#xp-feature-impure-derivations) experimental feature is enabled. - If it is pure, it must be floating. - Pure, fixed content-addressing derivations are not supported > There is no use for this forth combination. The sole purpose of an output's store path being fixed is to support the derivation being impure. ## [Content-addressing derivation outputs](https://docharvest.github.io/docs/nix/store/derivation/outputs/content-address/) Contents nix Content-addressing derivation outputs Nix Content-addressing derivation outputs The content-addressing of an output only depends on that store object itself, not any other information external (such has how it was made, when it was made, etc.). As a consequence, a store object will be content-addressed the same way regardless of whether it was manually inserted into the store, outputted by some derivation, or outputted by a some other derivation. The output spec for a content-addressed output must contains the following field: - _method_: how the data of the store object is digested into a content address The possible choices of _method_ are described in the [section on content-addressing store objects](@docroot@/store/store-object/content-address.md). Given the method, the output's name (computed from the derivation name and output spec mapping as described above), and the data of the store object, the output's store path will be computed as described in that section. ## Fixed-output content-addressing {#fixed} In this case the content address of the _fixed_ in advanced by the derivation itself. In other words, when the derivation has finished [building](@docroot@/store/building.md), and the provisional output' content-address is computed as part of the process to turn it into a _bona fide_ store object, the calculated content address must match that given in the derivation, or the build of that derivation will be deemed a failure. The output spec for an output with a fixed content addresses additionally contains: - _hash_, the hash expected from digesting the store object's file system objects. This hash may be of a freely-chosen hash algorithm (that Nix supports) > **Design note** > > In principle, the output spec could also specify the references the store object should have, since the references and file system objects are equally parts of a content-addressed store object proper that contribute to its content-addressed. However, at this time, the references are not done because all fixed content-addressed outputs are required to have no references (including no self-reference). > > Also in principle, rather than specifying the references and file system object data with separate hashes, a single hash that constraints both could be used. This could be done with the final store path's digest, or better yet, the hash that will become the store path's digest before it is truncated. > > These possible future extensions are included to elucidate the core property of fixed-output content addressing --- that all parts of the output must be cryptographically fixed with one or more hashes --- separate from the particulars of the currently-supported store object content-addressing schemes. ### Design rationale What is the purpose of fixing an output's content address in advanced? In abstract terms, the answer is carefully controlled impurity. Unlike a regular derivation, the [builder](@docroot@/store/derivation/index.md#builder) executable of a derivation that produced fixed outputs has access to the network. The outputs' guaranteed content-addresses are supposed to mitigate the risk of the builder being given these capabilities; regardless of what the builder does _during_ the build, it cannot influence downstream builds in unanticipated ways because all information it passed downstream flows through the outputs whose content-addresses are fixed. In concrete terms, the purpose of this feature is fetching fixed input data like source code from the network. For example, consider a family of "fetch URL" derivations. These derivations download files from given URL. To ensure that the downloaded file has not been modified, each derivation must also specify a cryptographic hash of the file. For example, ``` { "outputs: { "out": { "method": "nar", "hashAlgo": "sha256", "hash: "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465", }, }, "env": { "url": "http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz" // ... }, // ... } ``` It sometimes happens that the URL of the file changes, e.g., because servers are reorganised or no longer available. In these cases, we then must update the call to `fetchurl`, e.g., ``` "env": { - "url": "http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz" + "url": "ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz" // ... }, ``` If a `fetchurl` derivation's outputs were [input-addressed](./input-address.md), the output paths of the derivation and of _all derivations depending on it_ would change. For instance, if we were to change the URL of the Glibc source distribution in Nixpkgs (a package on which almost all other packages depend on Linux) massive rebuilds would be needed. This is unfortunate for a change which we know cannot have a real effect as it propagates upwards through the dependency graph. For content-addressed outputs (fixed or floating), on the other hand, the outputs' store path only depends on the derivation's name, data, and the `method` of the outputs' specs. The rest of the derivation is ignored for the purpose of computing the output path. > **History Note** > > Fixed content-addressing is especially important both today and historically as the _only_ form of content-addressing that is stabilized. This is why the rationale above contrasts it with [input addressing](./input-address.md). ## (Floating) Content-Addressing {#floating} > **Warning** This is part of an [experimental feature](@docroot@/development/experimental-features.md). > > To use this type of output addressing, you must enable the [`ca-derivations`](@docroot@/development/experimental-features.md#xp-feature-ca-derivations) experimental feature. For example, in [nix.conf](@docroot@/command-ref/conf-file.md) you could add: > > ``` > extra-experimental-features = ca-derivations > ``` With this experimemental feature enabled, derivation outputs can also be content-addressed _without_ fixing in the output spec what the outputs' content address must be. ### Purity Because the derivation output is not fixed (just like with [input addressing](./input-address.md)), the [builder](@docroot@/store/derivation/index.md#builder) is not given any impure capabilities \[^purity\]. > **Configuration note** > > Strictly speaking, the extent to which sandboxing and deprivilaging is possible varies with the environment Nix is running in. Nix's configuration settings indicate what level of sandboxing is required or enabled. Builds of derivations will fail if they request an absence of sandboxing which is not allowed. Builds of derivations will also fail if the level of sandboxing specified in the configure exceeds what is possible in the given environment. > > (The "environment", in this case, consists of attributes such as the Operating System Nix runs atop, along with the operating-system-specific privileges that Nix has been granted. Because of how conventional operating systems like macos, Linux, etc. work, granting builders _fewer_ privileges may ironically require that Nix be run with _more_ privileges.) That said, derivations producing floating content-addressed outputs may declare their builders as impure (like the builders of derivations producing fixed outputs). This is provisionally supported as part of the [`impure-derivations`](@docroot@/development/experimental-features.md#xp-feature-impure-derivations) experimental feature. ### Compatibility negotiation Any derivation producing a floating content-addressed output implicitly requires the `ca-derivations` [system feature](@docroot@/command-ref/conf-file.md#conf-system-features). This prevents scheduling the building of the derivation on a machine without the experimental feature enabled. Even once the experimental feature is stabilized, this is still useful in order to be allow using remote builder running odler versions of Nix, or alternative implementations that do not support floating content addressing. ### Determinism In the earlier [discussion of how self-references are handled when content-addressing store objects](@docroot@/store/store-object/content-address.html#self-references), it was pointed out that methods of producing store objects ought to be deterministic regardless of the choice of provisional store path. For store objects produced by manually inserting into the store to create a store object, the "method of production" is an informally concept --- formally, Nix has no idea where the store object came from, and content-addressing is crucial in order to ensure that the derivation is _intrinsically_ tamper-proof. But for store objects produced by derivation, the "method is quite formal" --- the whole point of derivations is to be a formal notion of building, after all. In this case, we can elevate this informal property to a formal one. A _deterministic_ content-addressing derivation should produce outputs with the same content addresses: 1. Every time the builder is run This is because either the builder is completely sandboxed, or because all any remaining impurities that leak inside the build sandbox are ignored by the builder and do not influence its behavior. 2. Regardless of the choice of any provisional outputs paths Provisional store paths must be chosen for any output that has a self-reference. The choice of provisional store path can be thought of as an impurity, since it is an arbitrary choice. If provisional outputs paths are deterministically chosen, we are in the first branch of part (1). The builder the data it produces based on it in arbitrary ways, but this gets us closer to [input addressing](./input-address.md). Deterministically choosing the provisional path may be considered "complete sandboxing" by removing an impurity, but this is unsatisfactory If provisional outputs paths are randomly chosen, we are in the second branch of part (1). The builder _must_ not let the random input affect the final outputs it produces, and multiple builds may be performed and the compared in order to ensure that this is in fact the case. ### Floating versus Fixed While the distinction between content- and input-addressing is one of _mechanism_, the distinction between fixed and floating content addressing is more one of _policy_. A fixed output that passes its content address check is just like a floating output. It is only in the potential for that check to fail that they are different. > **Design Note** > > In a future world where floating content-addressing is also stable, we in principle no longer need separate [fixed](#fixed) content-addressing. Instead, we could always use floating content-addressing, and separately assert the precise value content address of a given store object to be used as an input (of another derivation). A stand-alone assertion object of this sort is not yet implemented, but its possible creation is tracked in [issue #11955](https://github.com/NixOS/nix/issues/11955). > > In the current version of Nix, fixed outputs which fail their hash check are still registered as valid store objects, just not registered as outputs of the derivation which produced them. This is an optimization that means if the wrong output hash is specified in a derivation, and then the derivation is recreated with the right output hash, derivation does not need to be rebuilt — avoiding downloading potentially large amounts of data twice. This optimisation prefigures the design above: If the output hash assertion was removed outside the derivation itself, Nix could additionally not only register that outputted store object like today, but could also make note that derivation did in fact successfully download some data. For example, for the "fetch URL" example above, making such a note is tantamount to recording what data is available at the time of download at the given URL. It would only be when Nix subsequently tries to build something with that (refining our example) downloaded source code that Nix would be forced to check the output hash assertion, preventing it from e.g. building compromised malware. > > Recapping, Nix would > > 1. successfully download data > 2. insert that data into the store > 3. associate (presumably with some sort of expiration policy) the downloaded data with the derivation that downloaded it > > But only use the downloaded store object in subsequent derivations that depended upon the assertion if the assertion passed. > > This possible future extension is included to illustrate this distinction: ## [Input-addressing derivation outputs](https://docharvest.github.io/docs/nix/store/derivation/outputs/input-address/) Contents nix Input-addressing derivation outputs Nix Input-addressing derivation outputs "Input addressing" means the address the store object by the _way it was made_ rather than _what it is_. That is to say, an input-addressed output's store path is a function not of the output itself, but of the derivation that produced it. Even if two store paths have the same contents, if they are produced in different ways, and one is input-addressed, then they will have different store paths, and thus guaranteed to not be the same store object. ## Modulo content addressed derivation outputs {#hash-quotient-drv} A naive implementation of an output hash computation for input-addressed outputs would be to hash the derivation hash and output together. This clearly has the uniqueness properties we want for input-addressed outputs, but suffers from an inefficiency. Specifically, new builds would be required whenever a change is made to a fixed-output derivation, despite having provably no differences in the inputs to the new derivation compared to what it used to be. Concretely, this would cause a "mass rebuild" whenever any fetching detail changes, including mirror lists, certificate authority certificates, etc. To solve this problem, we compute output hashes differently, so that certain output hashes become identical. We call this concept quotient hashing, in reference to quotient types or sets. So how do we compute the [hash part](@docroot@/store/store-path.md#digest) of the output paths of an input-addressed derivation? This is done by the function `hashQuotientDerivation`, shown below. First, a word on inputs. `hashQuotientDerivation` is only defined on derivations whose [inputs](@docroot@/store/derivation/index.md#inputs) take the first-order form: ``` type ConstantPath = { path: StorePath; }; type FirstOrderOutputPath = { drvPath: StorePath; output: OutputName; }; type FirstOrderDerivingPath = ConstantPath | FirstOrderOutputPath; type Inputs = Set; ``` For the algorithm below, we adopt a derivation where the two types of (first order) derived paths are partitioned into two sets, as follows: ``` type Derivation = { // inputs: Set; // replaced inputSrcs: Set; // new instead inputDrvOutputs: Set; // new instead // ...other fields... }; ``` In the [currently-experimental](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) higher-order case where outputs of outputs are allowed as [deriving paths](@docroot@/store/derivation/index.md#deriving-path) and thus derivation inputs, derivations using that generalization are not valid arguments to this function. Those derivations must be (partially) [resolved](@docroot@/store/resolution.md) enough first, to the point where no such higher-order inputs remain. Then, and only then, can input addresses be assigned. ``` function hashQuotientDerivation(drv) -> Hash: assert(drv.outputs are input-addressed) drv′ ← drv with { inputDrvOutputs = ⋃( assert(drvPath is store path) case hashOutputsOrQuotientDerivation(readDrv(drvPath)) of drvHash : Hash → (drvHash.toBase16(), output) outputHashes : Map[String, Hash] → (outputHashes[output].toBase16(), "out") | (drvPath, output) ∈ drv.inputDrvOutputs ) } return hashSHA256(printDrv(drv′)) function hashOutputsOrQuotientDerivation(drv) -> Map[String, Hash] | Hash: if drv.outputs are content-addressed: return { outputName ↦ hashSHA256( "fixed:out:" + ca.printMethodAlgo() + ":" + ca.hash.toBase16() + ":" + ca.makeFixedOutputPath(drv.name, outputName)) | (outputName ↦ output) ∈ drv.outputs , ca = output.contentAddress // or get from build trace if floating } else: // drv.outputs are input-addressed return hashQuotientDerivation(drv) ``` ### `hashQuotientDerivation` We replace each element in the derivation's `inputDrvOutputs` using data from a call to `hashOutputsOrQuotientDerivation` on the `drvPath` of that element. When `hashOutputsOrQuotientDerivation` returns a single drv hash (because the input derivation in question is input-addressing), we simply swap out the `drvPath` for that hash, and keep the same output name. When `hashOutputsOrQuotientDerivation` returns a map of content addresses per-output, we look up the output in question, and pair it with the output name `out`. The resulting pseudo-derivation (with hashes instead of store paths in `inputDrvs`) is then printed (in the ["ATerm" format](@docroot@/protocols/derivation-aterm.md)) and hashed, and this becomes the hash of the "quotient derivation". When calculating output hashes, `hashQuotientDerivation` is called on an almost-complete input-addressing derivation, which is just missing its input-addressed outputs paths. The derivation hash is then used to calculate output paths for each output. Those output paths can then be substituted into the almost-complete input-addressed derivation to complete it. > **Note** > > There may be an unintentional deviation from specification currently implemented in the `(outputHashes[output].toBase16(), "out")` case. This is not fatal because the deviation would only apply for content-addressing derivations with more than one output, and that only occurs in the floating case, which is [experimental](@docroot@/development/experimental-features.md#xp-feature-ca-derivations). Once this bug is fixed, this note will be removed. ### `hashOutputsOrQuotientDerivation` How does `hashOutputsOrQuotientDerivation` in turn work? It consists of two main cases, based on whether the outputs of the derivation are to be input-addressed or content-addressed. #### Input-addressed outputs case In the input-addressed case, it just calls `hashQuotientDerivation`, and returns that derivation hash. This makes `hashQuotientDerivation` and `hashOutputsOrQuotientDerivation` mutually-recursive. > **Note** > > In this case, `hashQuotientDerivation` is being called on a _complete_ input-addressing derivation that already has its output paths calculated. The `inputDrvs` substitution takes place anyways. #### Content-addressed outputs case If the outputs are [content-addressed](./content-address.md), then it computes a hash for each output derived from the content-address of that output. > **Note** > > In the [fixed](./content-address.md#fixed) content-addressing case, the outputs' content addresses are statically specified in advance, so this always just works. (The fixed case is what the pseudo-code shows.) > > In the [floating](./content-address.md#floating) case, the content addresses are not specified in advance. This is what the "or get from [build trace](@docroot@/store/build-trace.md) if floating" comment refers to. In this case, the algorithm is _stuck_ until the input in question is built, and we know what the actual contents of the output in question is. > > That is OK however, because there is no problem with delaying the assigning of input addresses (which, remember, is what `hashQuotientDerivation` is ultimately for) until all inputs are known. ### Performance The recursion in the algorithm is potentially inefficient: it could call itself once for each path by which a subderivation can be reached, i.e., `O(V^k)` times for a derivation graph with `V` derivations and with out-degree of at most `k`. In the actual implementation, [memoisation](https://en.wikipedia.org/wiki/Memoization) is used to reduce this cost to be proportional to the total number of `inputDrvOutputs` encountered. ### Semantic properties _See [this chapter's appendix](@docroot@/store/math-notation.md) on grammar and metavariable conventions._ In essence, `hashQuotientDerivation` partitions input-addressing derivations into equivalence classes: every derivation in that equivalence class is mapped to the same derivation hash. We can characterize this equivalence relation directly, by working bottom up. We start by defining an equivalence relation on first-order output deriving paths that refer content-addressed derivation outputs. Two such paths are equivalent if they refer to the same store object: \\\[ \\begin{prooftree} \\AxiomC{$d\_1$ is content-addressing} \\AxiomC{$d\_2$ is content-addressing} \\AxiomC{$ {}^\*(\\text{path}(d\_1), o\_1) = {}^\*(\\text{path}(d\_2), o\_2) $} \\TrinaryInfC{$(\\text{path}(d\_1), o\_1) \\,\\sim\_{\\mathrm{CA}}\\, (d\_2, o\_2)$} \\end{prooftree} \\\] where \\({}^\*(s, o)\\) denotes the store object that the output deriving path refers to. We will also need the following construction to lift any equivalence relation on \\(X\\) to an equivalence relation on (finite) sets of \\(X\\) (in short, \\(\\mathcal{P}(X)\\)): \\\[ \\begin{prooftree} \\AxiomC{$\\forall a \\in A. \\exists b \\in B. a \\,\\sim\_X\\, b$} \\AxiomC{$\\forall b \\in B. \\exists a \\in A. b \\,\\sim\_X\\, a$} \\BinaryInfC{$A \\,\\sim\_{\\mathcal{P}(X)}\\, B$} \\end{prooftree} \\\] Now we can define the equivalence relation \\(\\sim\_\\mathrm{IA}\\) on input-addressed derivation outputs. Two input-addressed outputs are equivalent if their derivations are equivalent (via the yet-to-be-defined \\(\\sim\_{\\mathrm{IADrv}}\\) relation) and their output names are the same: \\\[ \\begin{prooftree} \\AxiomC{$d\_1$ is input-addressing} \\AxiomC{$d\_2$ is input-addressing} \\AxiomC{$d\_1 \\,\\sim\_{\\mathrm{IADrv}}\\, d\_2$} \\AxiomC{$o\_1 = o\_2$} \\QuaternaryInfC{$(\\text{path}(d\_1), o\_1) \\,\\sim\_{\\mathrm{IA}}\\, (\\text{path}(d\_2), o\_2)$} \\end{prooftree} \\\] And now we can define \\(\\sim\_{\\mathrm{IADrv}}\\). Two input-addressed derivations are equivalent if their content-addressed inputs are equivalent, their input-addressed inputs are also equivalent, and they are otherwise equal: \\\[ \\begin{prooftree} \\alwaysNoLine \\AxiomC{$ \\mathrm{caInputs}(d\_1) \\,\\sim\_{\\mathcal{P}(\\mathrm{CA})}\\, \\mathrm{caInputs}(d\_2) $} \\AxiomC{$ \\mathrm{iaInputs}(d\_1) \\,\\sim\_{\\mathcal{P}(\\mathrm{IA})}\\, \\mathrm{iaInputs}(d\_2) $} \\BinaryInfC{$ d\_1\\left\[\\mathrm{inputDrvOutputs} := \\{\\}\\right\] = d\_2\\left\[\\mathrm{inputDrvOutputs} := \\{\\}\\right\] $} \\alwaysSingleLine \\UnaryInfC{$d\_1 \\,\\sim\_{\\mathrm{IADrv}}\\, d\_2$} \\end{prooftree} \\\] where \\(\\mathrm{caInputs}(d)\\) returns the content-addressed inputs of \\(d\\) and \\(\\mathrm{iaInputs}(d)\\) returns the input-addressed inputs. > **Note** > > An astute reader might notice that that nowhere does `inputSrcs` enter into these definitions. That means that replacing an input derivation with its outputs directly added to `inputSrcs` always results in a derivation in a different equivalence class, despite the resulting input closure (as would be mounted in the store at build time) being the same. [Issue #9259](https://github.com/NixOS/nix/issues/9259) is about creating a coarser equivalence relation to address this. > > \\(\\sim\_\\mathrm{Drv}\\) from [derivation resolution](@docroot@/store/resolution.md) is such an equivalence relation. It is coarser than this one: any two derivations which are "'hash quotient derivation'-equivalent" (\\(\\sim\_\\mathrm{IADrv}\\)) are also "resolution-equivalent" (\\(\\sim\_\\mathrm{Drv}\\)). It also relates derivations whose `inputDrvOutputs` have been rewritten into `inputSrcs`. ## [File System Object](https://docharvest.github.io/docs/nix/store/file-system-object/) Contents nix File System Object Nix File System Object Nix uses a simplified model of the file system, which consists of file system objects. Every file system object is one of the following: - \[**Regular File**\]{#regular} - A possibly empty sequence of bytes for contents - A single boolean representing the [executable](https://en.m.wikipedia.org/wiki/File-system_permissions#Permissions) permission - \[**Directory**\]{#directory} Mapping of names to child file system objects - \[**Symbolic link**\]{#symlink} An arbitrary string, known as the _target_ of the symlink. In general, Nix does not assign any semantics to symbolic links. Certain operations however, may make additional assumptions and attempt to use the target to find another file system object. > See [the Wikipedia article on symbolic links](https://en.m.wikipedia.org/wiki/Symbolic_link) for background information if you are unfamiliar with this Unix concept. File system objects and their children form a tree. A bare file or symlink can be a root file system object. Nix does not encode any other file system notions such as [hard links](https://en.m.wikipedia.org/wiki/Hard_link), [permissions](https://en.m.wikipedia.org/wiki/File-system_permissions), timestamps, or other metadata. ## Examples of file system objects A plain file: ``` 50 B, executable: false ``` An executable file: ``` 122 KB, executable: true ``` A symlink: ``` -> /usr/bin/sh ``` A directory with contents: ``` ├── bin │   └── hello: 35 KB, executable: true └── share ├── info │   └── hello.info: 36 KB, executable: false └── man └── man1 └── hello.1.gz: 790 B, executable: false ``` A directory that contains a symlink and other directories: ``` ├── bin -> share/go/bin ├── nix-support/ └── share/ ``` ## [Content-Addressing File System Objects](https://docharvest.github.io/docs/nix/store/file-system-object/content-address/) Contents nix Content-Addressing File System Objects Nix Content-Addressing File System Objects For many operations, Nix needs to calculate [a content addresses](@docroot@/glossary.md#gloss-content-address) of [a file system object](../file-system-object.md) (FSO). Usually this is needed as part of [content addressing store objects](../store-object/content-address.md), since store objects always have a root file system object. But some command-line utilities also just work on "raw" file system objects, not part of any store object. Every content addressing scheme Nix uses ultimately involves feeding data into a [hash function](https://en.wikipedia.org/wiki/Hash_function), and getting back an opaque fixed-size digest which is deemed a content address. The various _methods_ of content addressing thus differ in how abstract data (in this case, a file system object and its descendants) are fed into the hash function. ## Serialising File System Objects { #serial } The simplest method is to serialise the entire file system object tree into a single binary string, and then hash that binary string, yielding the content address. In this section we describe the currently-supported methods of serialising file system objects. ### Flat { #serial-flat } A single file object can just be hashed by its contents. This is not enough information to encode the fact that the file system object is a file, but if we _already_ know that the FSO is a single non-executable file by other means, it is sufficient. Because the hashed data is just the raw file, as is, this choice is good for compatibility with other systems. For example, Unix commands like `sha256sum` or `sha1sum` will produce hashes for single files that match this. ### Nix Archive (NAR) { #serial-nix-archive } For the other cases of [file system objects](../file-system-object.md), especially directories with arbitrary descendants, we need a more complex serialisation format. Examples of such serialisations are the ZIP and TAR file formats. However, for our purposes these formats have two problems: - They do not have a canonical serialisation, meaning that given an FSO, there can be many different serialisations. For instance, TAR files can have variable amounts of padding between archive members; and some archive formats leave the order of directory entries undefined. This would be bad because we use serialisation to compute cryptographic hashes over file system objects, and for those hashes to be useful as a content address or for integrity checking, uniqueness is crucial. Otherwise, correct hashes would report false mismatches, and the store would fail to find the content. - They store more information than we have in our notion of FSOs, such as time stamps. This can cause FSOs that Nix should consider equal to hash to different values on different machines, just because the dates differ. - As a practical consideration, the TAR format is the only truly universal format in the Unix environment. It has many problems, such as an inability to deal with long file names and files larger than 2^33 bytes. Current implementations such as GNU Tar work around these limitations in various ways. For these reasons, Nix has its very own archive format—the Nix Archive (NAR) format, which is carefully designed to avoid the problems described above. The exact specification of the Nix Archive format is in [specified here](../../protocols/nix-archive/index.md). ## Content addressing File System Objects beyond a single serialisation pass Serialising the entire tree and then hashing that binary string is not the only option for content addressing, however. Another technique is that of a [Merkle graph](https://en.wikipedia.org/wiki/Merkle_tree), where previously computed hashes are included in subsequent byte strings to be hashed. In particular, the Merkle graphs can match the original graph structure of file system objects: we can first hash (serialised) child file system objects, and then hash parent objects using the hashes of their children in the serialisation (to be hashed) of the parent file system objects. Currently, there is one such Merkle DAG content addressing method supported. ### Git ([experimental](@docroot@/development/experimental-features.md#xp-feature-git-hashing)) { #git } > **Warning** > > This method is part of the [`git-hashing`](@docroot@/development/experimental-features.md#xp-feature-git-hashing) experimental feature. Git's file system model is very close to Nix's, and so Git's content addressing method is a pretty good fit. Just as with regular Git, files and symlinks are hashed as git "blobs", and directories are hashed as git "trees". However, one difference between Nix's and Git's file system model needs special treatment. Plain files, executable files, and symlinks are not differentiated as distinctly addressable objects, but by their context: by the directory entry that refers to them. That means so long as the root object is a directory, there is no problem: every non-directory object is owned by a parent directory, and the entry that refers to it provides the missing information. However, if the root object is not a directory, then we have no way of knowing which one of an executable file, non-executable file, or symlink it is supposed to be. In response to this, we have decided to treat a bare file as non-executable file. This is similar to do what we do with [flat serialisation](#serial-flat), which also lacks this information. To avoid an address collision, attempts to hash a bare executable file or symlink will result in an error (just as would happen for flat serialisation also). Thus, Git can encode some, but not all of Nix's "File System Objects", and this sort of content-addressing is likewise partial. In the future, we may support a Git-like hash for such file system objects, or we may adopt another Merkle DAG format which is capable of representing all Nix file system objects. ## [Exposing File System Objects in real operating system file systems](https://docharvest.github.io/docs/nix/store/file-system-object/os-file-system/) Contents nix Exposing File System Objects in real operating system file systems Nix Exposing File System Objects in real operating system file systems Nix's [file system object](../file-system-object.md) data model is minimal. All the various other bits and pieces of real world filesystem interfaces, such as [extended file attributes](https://en.wikipedia.org/wiki/Extended_file_attributes), are specifically ignored to reduce our interface surface and the reproducibility issues associated with a larger interface. In the view of Nix's developers, the types of simple, fine-grained batch jobs (typically, building software) that Nix specializes in simply don't benefit enough from that extra complexity for it to be worth the costs of supporting it. But to actually be used by software, file system objects need to be made available through the operating system's file system. This is sometimes called "mounting" or "exposing" the file system object, though do note it may or may not be implemented with what the operating system calls "mounting". ## Metadata normalization File systems typically contain other metadata that is outside Nix's data model. To avoid this other metadata being a side channel and source of nondeterminism, Nix is careful to normalize to fixed values. For example, on Unix, the following metadata normalization occurs: - The creation and last modification timestamps on all files are set to Unix Epoch 1s (00:00:01 1/1/1970 UTC) - The group is set to the [default group](@docroot@/command-ref/conf-file.md#conf-build-users-group) - The Unix mode of the file to 0444 or 0555 (i.e., read-only, with execute permission enabled if the file was originally executable). - Any possible `setuid` and `setgid` bits are cleared. > **Note** > > `setuid` and `setgid` programs are not currently supported by Nix. These special file system permissions are in general a security footgun, and with data owned by different users in different stores, it would especially be a hazard when copying store objects between stores. > > This restriction has not proved to be onerous in practice. For example, NixOS uses so called setuid-wrappers which are outside the store. > **Explanation** > > As discussed before, Nix essentially shares its file system object data model with other tools like Git. But those tools tend to ignore this metadata in both directions --- when reading files, like Nix, but when writing files, timestamps are set organically, and the user is free to set other special permissions (`setuid`, `setgid`, sticky, etc.) however they like. Normalizing, and not just ignoring, this metadata is therefore what distinguishes Nix from these other tools more than the file system object data model itself. > > Nix's approach is motivated by deterministic building. Whereas Git can assume that humans running commands will simply ignore timestamps etc. as appropriate, understanding they are local and ephemeral, Nix aims to run software that was not necessarily designed with Nix in mind, and is unaware of whatever sandboxing/virtualization is in place. ## [Appendix: Math notation](https://docharvest.github.io/docs/nix/store/math-notation/) Contents nix Appendix: Math notation Nix Appendix: Math notation A few times in this manual, formal "proof trees" are used for [natural deduction](https://en.wikipedia.org/wiki/Natural_deduction)\-style definition of various [relations](https://en.wikipedia.org/wiki/Relation_\(mathematics\)). The following grammar and assignment of metavariables to syntactic categories is used in these sections. \\begin{align} s, t &\\in \\text{store-path} \\\\ o &\\in \\text{output-name} \\\\ i, p &\\in \\text{deriving-path} \\\\ d &\\in \\text{derivation} \\end{align} \\begin{align} \\text{deriving-path} \\quad p &::= s \\mid (p, o) \\end{align} ## [Derivation Resolution](https://docharvest.github.io/docs/nix/store/resolution/) Contents nix Derivation Resolution Nix Derivation Resolution _See [this chapter's appendix](@docroot@/store/math-notation.md) on grammar and metavariable conventions._ To _resolve_ a derivation is to replace its [inputs](@docroot@/store/derivation/index.md#inputs) with the simplest inputs — plain store paths — that denote the same store objects. Derivations that only have store paths as inputs are likewise called _resolved derivations_. (They are called that whether they are in fact the output of derivation resolution, or just made that way without non-store-path inputs to begin with.) ## Input Content Equivalence of Derivations [Deriving paths](@docroot@/store/derivation/index.md#deriving-path) intentionally make it possible to refer to the same [store object](@docroot@/store/store-object.md) in multiple ways. This is a consequence of content-addressing, since different derivations can produce the same outputs, and the same data can also be manually added to the store. This is also a consequence even of input-addressing, as an output can be referred to by derivation and output name, or directly by its [computed](./derivation/outputs/input-address.md) store path. Since dereferencing deriving paths is thus not injective, it induces an equivalence relation on deriving paths. Let's call this equivalence relation \\(\\sim\\), where \\(p\_1 \\sim p\_2\\) means that deriving paths \\(p\_1\\) and \\(p\_2\\) refer to the same store object. **Content Equivalence**: Two deriving paths are equivalent if they refer to the same store object: \\\[ \\begin{prooftree} \\AxiomC{${}^\*p\_1 = {}^\*p\_2$} \\UnaryInfC{$p\_1 \\,\\sim\_\\mathrm{DP}\\, p\_2$} \\end{prooftree} \\\] where \\({}^\*p\\) denotes the store object that deriving path \\(p\\) refers to. This also induces an equivalence relation on sets of deriving paths: \\\[ \\begin{prooftree} \\AxiomC{$\\{ {}^\*p | p \\in P\_1 \\} = \\{ {}^\*p | p \\in P\_2 \\}$} \\UnaryInfC{$P\_1 \\,\\sim\_{\\mathcal{P}(\\mathrm{DP})}\\, P\_2$} \\end{prooftree} \\\] **Input Content Equivalence**: This, in turn, induces an equivalence relation on derivations: two derivations are equivalent if their inputs are equivalent, and they are otherwise equal: \\\[ \\begin{prooftree} \\AxiomC{$\\mathrm{inputs}(d\_1) \\,\\sim\_{\\mathcal{P}(\\mathrm{DP})}\\, \\mathrm{inputs}(d\_2)$} \\AxiomC{$ d\_1\\left\[\\mathrm{inputs} := \\{\\}\\right\] = d\_2\\left\[\\mathrm{inputs} := \\{\\}\\right\] $} \\BinaryInfC{$d\_1 \\,\\sim\_\\mathrm{Drv}\\, d\_2$} \\end{prooftree} \\\] Derivation resolution always maps derivations to input-content-equivalent derivations. ## Resolution relation Dereferencing a derived path — \\({}^\*p\\) above — was just introduced as a black box. But actually it is a multi-step process of looking up build results in the [build trace](@docroot@/store/build-trace.md) that itself depends on resolving the lookup keys. Resolution is thus a recursive multi-step process that is worth diagramming formally. We can do this with a small-step binary transition relation; let's call it \\(\\rightsquigarrow\\). We can then conclude dereferenced equality like this: \\\[ \\begin{prooftree} \\AxiomC{$p\_1 \\rightsquigarrow^\* p$} \\AxiomC{$p\_2 \\rightsquigarrow^\* p$} \\BinaryInfC{${}^\*p\_1 = {}^\*p\_2$} \\end{prooftree} \\\] I.e. by showing that both original items resolve (over 0 or more small steps, hence the \\({}^\*\\)) to the same exact item. With this motivation, let's now formalize a [small-step](https://en.wikipedia.org/wiki/Operational_semantics#Small-step_semantics) system of reduction rules for resolution. ### Formal rules ### \\(\\text{resolved}\\) unary relation \\\[ \\begin{prooftree} \\AxiomC{$s \\in \\text{store-path}$} \\UnaryInfC{$s$ resolved} \\end{prooftree} \\\] \\\[ \\begin{prooftree} \\AxiomC{$\\forall i \\in \\mathrm{inputs}(d). i \\text{ resolved}$} \\UnaryInfC{$d$ resolved} \\end{prooftree} \\\] ### \\(\\rightsquigarrow\\) binary relation > **Remark** > > Actually, to be completely formal we would need to keep track of the build trace we are choosing to resolve against. > > We could do that by making \\(\\rightsquigarrow\\) a ternary relation, which would pass the build trace to itself until it finally uses it in that one rule. This would add clutter more than insight, so we didn't bother to write it. > > There are other options too, like saying the whole reduction rule system is parameterized on the build trace, essentially [currying](https://en.wikipedia.org/wiki/Currying) the ternary \\(\\rightsquigarrow\\) into a function from build traces to the binary relation written above. #### Core build trace lookup rule \\\[ \\begin{prooftree} \\AxiomC{$s \\in \\text{store-path}$} \\AxiomC{${}^\*s \\in \\text{derivation}$} \\AxiomC{${}^\*s$ resolved} \\AxiomC{$\\mathrm{build\\text{-}trace}\[s\]\[o\] = t$} \\QuaternaryInfC{$(s, o) \\rightsquigarrow t$} \\RightLabel{\\scriptsize output path resolution} \\end{prooftree} \\\] #### Inductive rules \\\[ \\begin{prooftree} \\AxiomC{$i \\rightsquigarrow i'$} \\AxiomC{$i \\in \\mathrm{inputs}(d)$} \\BinaryInfC{$d \\rightsquigarrow d\[i \\mapsto i'\]$} \\end{prooftree} \\\] \\\[ \\begin{prooftree} \\AxiomC{$d \\rightsquigarrow d'$} \\UnaryInfC{$(\\mathrm{path}(d), o) \\rightsquigarrow (\\mathrm{path}(d'), o)$} \\end{prooftree} \\\] \\\[ \\begin{prooftree} \\AxiomC{$p \\rightsquigarrow p'$} \\UnaryInfC{$(p, o) \\rightsquigarrow (p', o)$} \\end{prooftree} \\\] ### Properties Like all well-behaved evaluation relations, partial resolution is [_confluent_](https://en.wikipedia.org/wiki/Confluence_\(abstract_rewriting\)). Also, if we take the symmetric closure of \\(\\rightsquigarrow^\*\\), we end up with the equivalence relations of the previous section. Resolution respects content equivalence for deriving paths, and input content equivalence for derivations. > **Remark** > > We chose to define from scratch an "resolved" unary relation explicitly above. But it can also be defined as the normal forms of the \\(\\rightsquigarrow^\*\\) relation: > > \\\[ a \\text{ resolved} \\Leftrightarrow \\forall b. b \\rightsquigarrow^\* a \\Rightarrow b = a\\\] > > In prose, resolved terms are terms which \\(\\rightsquigarrow^\*\\) only relates on the left side to the same term on the right side; they are the terms which can be resolved no further. ## Partial versus Complete Resolution Similar to evaluation, we can also speak of _partial_ versus _complete_ derivation resolution. Partial derivation resolution is what we've actually formalized above with \\(\\rightsquigarrow^\*\\). Complete resolution is resolution ending in a resolved term (deriving path or derivation). (Which is a normal form of the relation, per the remark above.) With partial resolution, a derivation is related to equivalent derivations with the same or simpler inputs, but not all those inputs will be plain store paths. This is useful when the input refers to a floating content addressed output we have not yet built — we don't know what (content-address) store path will used for that derivation, so we are "stuck" trying to resolve the deriving path in question. (In the above formalization, this happens when the build trace is missing the keys we wish to look up in it.) Complete resolution is a _functional_ relation, i.e. values on the left are uniquely related with values on the right. It is not however, a _total_ relation (in general, assuming arbitrary build traces). This is discussed in the next section. ## Termination For static derivations graphs, complete resolution is indeed total, because it always terminates for all inputs. (A relation that is both total and functional is a function.) For [dynamic](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) derivation graphs, however, this is not the case — resolution is not guaranteed to terminate. The issue isn't rewriting deriving paths themselves: a single rewrite to normalize an output deriving path to a constant one always exists, and always proceeds in one step. The issue is that dynamic derivations (i.e. those that are filled-in the graph by a previous resolution) may have more transitive dependencies than the original derivation. > **Example** > > Suppose we have this deriving path > > ``` > { > "drvPath": { > "drvPath": "...-foo.drv", > "output": "bar.drv" > }, > "output": "baz" > } > ``` > > and derivation `foo` is already resolved. When we resolve deriving path we'll end up with something like. > > ``` > { > "drvPath": "...-foo-bar.drv", > "output": "baz" > } > ``` > > So far is just an atomic single rewrite, with no termination issues. But the derivation `foo-bar` may have its _own_ dynamic derivation inputs. Resolution must resolve that derivation first before the above deriving path can finally be normalized to a plain `...-foo-bar-baz` store path. The important thing to notice is that while "build trace" _keys_ must be resolved. The _value_ those keys are mapped to have no such constraints. An arbitrary store object has no notion of being resolved or not. But, an arbitrary store object can be read back as a derivation (as will in fact be done in case for dynamic derivations / nested output deriving paths). And those derivations need _not_ be resolved. It is those dynamic non-resolved derivations which are the source of non-termination. By the same token, they are also the reason why dynamic derivations offer greater expressive power. ## [Secrets](https://docharvest.github.io/docs/nix/store/secrets/) Contents nix Secrets Nix Secrets The store is readable to all users on the system. For this reason, it is generally discouraged to allow secrets to make it into the store. Even on a single-user system, separate system users isolate services from each other and having secrets that all local users can read weakens that isolation. When using external store caches the secrets may end up there, and on multi-user systems the secrets will be available to all those users. Organize your derivations so that secrets are read from the filesystem (with appropriate access controls) at run time. Place the secrets on the filesystem manually or use a scheme that includes the secret in the store in encrypted form, and decrypts it adding the relevant access control on system activation. Several such schemes for NixOS can in the [comparison of secret managing schemes](https://wiki.nixos.org/wiki/Comparison_of_secret_managing_schemes) on the wiki. ## [Store Object](https://docharvest.github.io/docs/nix/store/store-object/) Contents nix Store Object Nix Store Object ## Store Object A Nix store is a collection of _store objects_ with _references_ between them. A store object consists of - A [file system object](./file-system-object.md) as data - A set of [store paths](./store-path.md) as references to store objects ### References Store objects can refer to both other store objects and themselves. References from a store object to itself are called _self-references_. Store objects and their references form a directed graph, where the store objects are the vertices, and the references are the edges. In particular, the edge corresponding to a reference is from the store object that contains the reference, and to the store object that the store path (which is the reference) refers to. References other than a self-reference must not form a cycle. The graph of references excluding self-references thus forms a [directed acyclic graph](@docroot@/glossary.md#gloss-directed-acyclic-graph). We can take the [transitive closure](https://en.wikipedia.org/wiki/Transitive_closure) of the references graph, in which any pair of store objects have an edge if a _path_ of one or more references exists from the first to the second object. (A single reference always forms a path which is one reference long, but longer paths may connect objects which have no direct reference between them.) The _requisites_ of a store object are all store objects reachable by paths of references which start with given store object's references. We can also take the [transpose graph](https://en.wikipedia.org/wiki/Transpose_graph) of the references graph, where we reverse the orientation of all edges. The _referrers_ of a store object are the store objects that reference it. One can also combine both concepts: taking the transitive closure of the transposed references graph. The _referrers closure_ of a store object are the store objects that can reach the given store object via paths of references. > **Note** > > Care must be taken to distinguish between the intrinsic and extrinsic properties of store objects. We can create graphs from the store objects in a store, but the contents of the store is not, in general fixed, and may instead change over time. > > - The references of a store object --- the set of store paths called the references --- is a field of a store object, and thus intrinsic by definition. Regardless of what store contains the store object in question, and what else that store may or may not contain, the references are the same. > > > - The requisites of a store object are almost intrinsic --- some store paths do not precisely refer to a unique single store object. Exactly what store object is being referenced, and what in turn _its_ references are, depends on the store in question. Different stores may disagree on what a given store path refers to. > > > > - The referrers of a store object are completely extrinsic, and depends solely on the store which contains that store object, not the store object itself. Other store objects which refer to the store object in question may be added or removed from the store. > > > ### Immutability Store objects are [immutable](https://en.wikipedia.org/wiki/Immutable_object): Once created, they do not change nor can any store object they reference be changed. > **Note** > > Stores which support atomically deleting multiple store objects allow more flexibility while still upholding this property. ### Closure property A store can only contain a store object if it also contains all the store objects it refers to. > **Note** > > The "closure property" isn't meant to prohibit, for example, [lazy loading](https://en.wikipedia.org/wiki/Lazy_loading) of store objects. However, the "closure property" and immutability in conjunction imply that any such lazy loading ought to be deterministic. ### Store Object Metadata {#metadata} [Store implementations](@docroot@/store/types/index.md) currently associate more information than described above with a store object. Quite arguably some of this information doesn't belong here, because it conflates concerns. For details see the [store object info](@docroot@/protocols/json/store-object-info.md) JSON format or the [narinfo](@docroot@/protocols/binary-cache/narinfo.md) format. ## [Content-Addressing Store Objects](https://docharvest.github.io/docs/nix/store/store-object/content-address/) Contents nix Content-Addressing Store Objects Nix Content-Addressing Store Objects Just [like](../file-system-object/content-address.md) [File System Objects](../file-system-object.md), [Store Objects](../store-object.md) can also be [content-addressed](@docroot@/glossary.md#gloss-content-address), unless they are [input-addressed](@docroot@/glossary.md#gloss-input-addressed-store-object). For store objects, the content address we produce will take the form of a [Store Path](../store-path.md) rather than regular hash. In particular, the content-addressing scheme will ensure that the digest of the store path is solely computed from the - file system object graph (the root one and its children, if it has any) - references - [store directory path](../store-path.md#store-directory-path) - name of the store object, and not any other information, which would not be an intrinsic property of that store object. For the full specification of the algorithms involved, see the [specification of store path digests](@docroot@/protocols/store-path.md). ## Content addressing each part of a store object ### File System Objects With all currently-supported store object content-addressing methods, the file system object is always [content-addressed](../file-system-object/content-address.md) first, and then that hash is incorporated into content address computation for the store object. ### References #### References to other store objects With all currently supported store object content addressing methods, other objects are referred to by their regular (string-encoded-) [store paths](../store-path.md). #### Self-references Self-references however cannot be referred to by their path, because we are in the midst of describing how to compute that path! > The alternative would require finding as hash function fixed point, i.e. the solution to an equation in the form > > ``` > digest = hash(..... || digest || ....) > ``` > > which is computationally infeasible. As far as we know, this is equivalent to finding a hash collision. Instead we have a "has self-reference" boolean, which ends up affecting the digest: In all currently-supported store object content-addressing methods, when hashing the file system object data, any occurrence of store object's own store path in the digested data is replaced with a [sentinel value](https://en.wikipedia.org/wiki/Sentinel_value). The hashes of these modified input streams are used instead. When validating the content address of a store object after the fact, the above process works as written. However, when first creating the store object we don't know the store object's store path, as explained just above. We therefore, strictly speaking, do not know what value we will be replacing with the sentinel value in the inputs to hash functions. What instead happens is that the provisional store object --- the data from which we wish to create a store object --- is paired with a provisional "scratch" store path (that presumably was chosen when the data was created). That provisional store path is instead what is replaced with the sentinel value, rather than the final store object which we do not yet know. > **Design note** > > It is an informal property of content-addressed store objects that the choice of provisional store path should not matter. In other words, if a provisional store object is prepared in the same way except for the choice of provision store path, the provisional data need not be identical. But, after the sentinel value is substituted in place of each provisional store object's provision store path, the final so-normalized data _should_ be identical. > > If, conversely, the data after this normalization process is still different, we'll compute a different content-address. The method of preparing the provisional self-referenced data has _failed_ to be deterministic in the sense of not _leaking_ the choice of provisional store path --- a choice which is supposed to be arbitrary --- into the final store object. > > This property is informal at this stage, because we are just describing store objects, which have no formal notion of their origin. Without that, we cannot rigorously define what it would mean for the process to be insufficiently deterministic. When we cover [derivations](@docroot@/store/derivation/index.md), we will be able to make this a formal property — not of content-addressed store objects themselves, but of the derivations that _produce_ them. ### Name and Store Directory These two items affect the digest in a way that is standard for store path digest computations and not specific to content-addressing. Consult the [specification of store path digests](@docroot@/protocols/store-path.md) for further details. ## Content addressing Methods For historical reasons, we don't support all features in all combinations. Each currently supported method of content addressing chooses a single method of file system object hashing, and may offer some restrictions on references. The names and store directories are unrestricted however. ### Flat { #method-flat } This uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing. References are not supported: store objects with flat hashing _and_ references can not be created. ### Text { #method-text } This also uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing. References to other store objects are supported, but self-references are not. This is the only store-object content-addressing method that is not named identically with a corresponding file system object method. It is somewhat obscure, mainly used for "drv files" (derivations serialized as store objects in their ["ATerm" file format](@docroot@/protocols/derivation-aterm.md)). Prefer another method if possible. ### Nix Archive { #method-nix-archive } This uses the corresponding [Nix Archive](../file-system-object/content-address.md#serial-nix-archive) method of file system object content addressing. References (to other store objects and self-references alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise. ### Git { #method-git } > **Warning** > > This method is part of the [`git-hashing`](@docroot@/development/experimental-features.md#xp-feature-git-hashing) experimental feature. This uses the corresponding [Git](../file-system-object/content-address.md#git) method of file system object content addressing. References are not supported. Only SHA-1 is supported at this time. If [SHA-256-based Git](https://git-scm.com/docs/hash-function-transition) becomes more widespread, this restriction will be revisited. ## [Store Path and Store Directory](https://docharvest.github.io/docs/nix/store/store-path/) Contents nix Store Path and Store Directory Nix Store Path and Store Directory Nix's [store object](./store-object.md) and [file system object](./file-system-object.md) data models are minimal and abstract. But to actually be used by software, store objects need to be made available through the operating system's file system. This is done by exposing all the store objects in a single _[store directory](#store-directory-path)_. Every entry in that directory is a _[store path base name](#base-name)_ pointing to a store object. Store objects exposed in this way can then be referenced by _[store paths](#store-path)_. ## Store Path Base Name {#base-name} Nix implements references to store objects as _store path base names_. Think of a store path base name as an [opaque](https://en.m.wikipedia.org/wiki/Opaque_data_type), [unique identifier](https://en.m.wikipedia.org/wiki/Unique_identifier): The only way to obtain a store path base name is by adding or building store objects. A store path base name will always reference exactly one store object. Store path base names are pairs of - A 20-byte \[digest\]{#digest} for identification - A symbolic \[name\]{#name} for people to read > **Example** > > - Digest: `q06x3jll2yfzckz2bzqak089p43ixkkq` > - Name: `firefox-33.1` A store path base name is rendered to a string as the concatenation of - Digest rendered in [Nix32](@docroot@/protocols/nix32.md), a variant of [Base32](https://en.wikipedia.org/wiki/Base32) (20 hash bytes become 32 ASCII characters) - Hyphen (`-`) - Name > **Example** > > ``` > q06x3jll2yfzckz2bzqak089p43ixkkq-firefox-33.1 > |------------------------------| |----------| > digest name > ``` Exactly how the digest is calculated depends on the type of store object being referenced. Store path digests are _supposed_ to be opaque, and so for most operations, it is not necessary to know the details. That said, the manual has a full [specification of store path digests](@docroot@/protocols/store-path.md). ## Store Directory Path Every [Nix store](./index.md) has a store directory path. This is an absolute, lexically canonical (not containing any `..`, `.`, or similar) path which points to the directory where all store objects are to be found. > **Note** > > The store directory defaults to `/nix/store`, but is in principle arbitrary. ## Store Path A store path is the pair of a store directory path and a [store path base name](#base-name). It is rendered to a file system path as the concatenation of - [Store directory path](#store-directory-path) (typically `/nix/store`) - Path separator (`/`) - The [store path base name](#base-name) > **Example** > > ``` > /nix/store/q06x3jll2yfzckz2bzqak089p43ixkkq-firefox-33.1 > |--------| |------------------------------| |----------| > store directory digest name > ``` When we have fixed a given store, or given store directory path (that all the stores in use share), the abstract syntax for store paths and the abstract syntax for store path base names coincide: the store directory path is known from context, so only the other two fields vary from one store path to the next. ## Exposing Store Objects in OS File Systems {#exposing} Not every store can be accessed through the file system. But if the store has a file system representation, the following should be true: - The store directory path is canonical: no prefix of the path (i.e. path of the first _n_ path segments) points to a symlink. In other words, the store directory can be looked up from the store directory path without following any symlinks. (This condition is a separate condition in addition to the "lexical canonicity" described above, which is a property of just the path itself. This (regular) "canonicity" is a property about the path and the filesystem it navigates jointly.) > **Note** > > The [`allow-symlinked-store`](@docroot@/command-ref/conf-file.md#conf-allow-symlinked-store) setting can be used to relax this requirement. - The store directory path in fact points to a directory. - The store directory contains, for every store object in the store, the [file system object](./file-system-object.md) of that store object at the (rendered) [store path base name](#base-name). The permissions and other metadata for these files in the store directory is in the normal form described in [Exposing in OS file systems](./file-system-object/os-file-system.md). The above properties mean that the following file accesses will work. Suppose we have a store available on the file system per the above rules, and `b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1` is the store path base name of a store object in that store. - Suppose that the store directory (path) is `/foo/bar`. Then, `/foo/bar/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1` exists and is the file system object of that store object. - Suppose that we don't know what the store directory path of the store is, but we do have a capability `storeDir` to the store directory on the file system. (This would be a "file descriptor" on Unix, or a "file handle" on Windows.) Then (using the Unix notation for this): ``` openat(storeDir, "b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1", O_NOFOLLOW) ``` will succeed (so long as the file system object is not a symlink), and the yielded capability will point to the file system object of that store object. (The behavior for symlinks is harder to specify because of limitations in POSIX.) ## Relocating store objects The inclusion of the store directory path in the full rendered store path means that the full rendered store path is not just derived from the referenced store object itself, but depends on the store that the store object is in. (And actually, all of the currently-supported ways of computing the digest of a store path also depend on the store directory path, as described in the [specification of store path digests](@docroot@/protocols/store-path.md). So this is also true even just for store path base names, in general.) It is therefore important to consider which store a given store object belongs to: Files in the store object can contain store paths, and processes may read these paths. Nix can only guarantee referential integrity if store paths do not cross store boundaries. One can only copy store objects to a different store if - The source and target stores' directories match or - The store object in question has no references, that is, contains no store paths One cannot copy a store object to a store with a different store directory. Instead, it has to be rebuilt, together with all its dependencies. It is in general not enough to replace the store directory string in file contents, as this may render executables unusable by invalidating their internal offsets or checksums.