install: configure and bootstrap synthetic.conf on darwin #3212

LnL7 · 2019-11-08T16:52:21Z

Here's some preliminary documentation for the apfs volume creation section.

Updated to create an apfs volume instead of using the symlink approach. This works on a clean install but I expect there are other corner cases that are not covered. More testing is probably required.

Using a symlink instead of a separate volume is the most straightforward approach which relies less on darwin specific tools. However it does have some disadvantages, especially if this has to be changed in the future. Also the new location should be avoided for official infrastructure (at least for a while) so installation using an apfs volume should probably also be implemented somewhere.

From my testing it looks like apfs.util works properly now, unless this breaks again a reboot can be bypassed for both mountpoints and symlinks.

qolii · 2019-11-09T22:50:10Z

Wait, this is viable? From following #2925, I thought the consensus was that nixpkgs currently has impurities in it that preclude using a (symlink to a) non-/nix store path. Did I misunderstand, or is the worry just overblown and this will work for "most people"?

LnL7 · 2019-11-10T00:01:40Z

There's a big difference between changing NIX_STORE_DIR and it's physical location. With a symlinked store it's possible for build results to leak the canonical location. While this isn't great, it's is only causes problems when sharing caches between hosts that use different locations. So if all macs use the same target there's no issue. It's a tradeoff, but I feel like it's better to keep the installer as simple as possible.

If we avoid this on the hydra infrastructure that build the official caches (at least for now) I think it won't be a problem anybody runs into in practice.

scripts/install-nix-from-closure.sh

lilyball · 2019-11-10T00:16:01Z

I am a bit concerned that what I download from Hydra wouldn't match what I build locally, e.g. someone already said llvm (or clang?) seems to do a realpath on the build location and will embed the symlink destination.

As long as apfs.util works, can we not use it for the volume approach?

Another potential benefit of the volume approach that's been mentioned is the fact that we can make it case-sensitive.

qolii · 2019-11-10T07:20:09Z

While this isn't great, it's is only causes problems when sharing caches between hosts that use different locations. So if all macs use the same target there's no issue.

Ahh, gotcha. I think I understand now. And as @lilyball says, this would mean that packages built with a "real" store path of /nix might be different to ones we build on a Mac with the symlinked store. But your point is, while they might not match, everything will still work?

LnL7 · 2019-11-10T09:33:03Z

Long term this wouldn't be a problem. I'm more concerned about the complexity of managing a separate volume and what could change/break there in the future.

domenkozar · 2019-11-10T22:39:03Z

cc @edolstra I tested this and it works, it would be great to have 2.3.2 release out once it's reviewed and merged.

dhess · 2019-11-10T23:20:54Z

So worst case, if the official macOS builders stay on pre-10.15 for awhile and keep building packages for a physical /nix store location, it just means that Macs that symlink their physical store location to /nix will miss the Hydra cache occasionally, correct?

In any case, I think this should be /opt/nix, because it follows the Linux FHS and therefore there's a better chance (though, still, admittedly slim) that Nix on Linux could adopt it someday, as well.

lilyball · 2019-11-11T00:24:19Z

Are the Hydra macOS builders guaranteed to run on the OS that we use as the minimum macOS target? If we ever run a 10.15 Hydra macOS builder while we still support pre-10.15, that means binary caches for packages that "see through" the symlink will then break on the older OSes.

mroi · 2019-11-11T12:12:47Z

I like this solution a lot. However, since I experimented with a symlinked store before, we may need to extend the purity checks in the ld wrapper. I had problems, where the absolute path of the store would sneak into a build process. ld would then complain, that a library from outside the store was linked against, while in fact it was located within the store.

Unfortunately, I do not remember what I was building back then. So this is more of a warning than a call for action right now.

arcnmx · 2019-11-13T16:43:44Z

Aside from the obvious problems regarding purity, it's worth noting here that I've so far encountered a few inconsistencies in nix from trying to switch to a symlink'd store path:

builtins.storePath appears to just not work at all?
nix-build /nix/store/something.drv gets confused
- (need to check if nix-store -r has the same problem?)

LnL7 · 2019-11-13T21:36:18Z

@mroi Things like that I what I was most afraid of, would be great if you had an example.

scripts/install-nix-from-closure.sh

mroi · 2019-11-13T21:52:18Z

@mroi Things like that I what I was most afraid of, would be great if you had an example.

I honestly do not remember. Sorry. If I find the time, I will try some things I usually build myself.

domenkozar · 2019-11-18T19:30:04Z

Note that we'll also need to fix #2407 in order to support macos + multi-user mode.

EDIT: already done in d4e51aa#diff-3f9ecc4aa9722ed5597a741e4cb91d3e

LnL7 · 2019-12-15T15:47:59Z

release.nix

@@ -176,11 +180,15 @@ let
            --absolute-names \
            --hard-dereference \
            --transform "s,$TMPDIR/install,$dir/install," \
+            --transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \


Not sure why only this file ends up in the wrong place.

scripts/install-nix-from-closure.sh

scripts/create-darwin-volume.sh

matthewbauer · 2019-12-16T19:29:15Z

Looks good besides a few small things. I think it would be good to get this in a Nix release pretty soon as we lose lots of potential users without a well-documented Catalina installer.

Tried on a fresh macOS using the following steps:

# on a pre-10.15 Nix install
$ git clone https://github.com/LnL7/nix
$ cd nix
$ git checkout darwin-10.15-install
$ nix build -f release.nix binaryTarball
$ cp result/*.tar.xz ~/nix.tar.xz
# Copy ~/nix.tar.xz to your Catalina macOS and open a terminal there
$ tar xf nix.tar.xz
$ ./nix-*/create-volume.sh
$ ./nix-*/install.sh --daemon

scripts/create-darwin-volume.sh

lilyball · 2019-12-16T23:37:41Z

scripts/create-darwin-volume.sh

+                break
+                ;;
+        esac
+        i=$((i+1))


This can be let i+=1

lilyball · 2019-12-16T23:38:31Z

scripts/create-darwin-volume.sh

+}
+
+apfs_volumes_for() {
+    disk=$1


Perhaps local

lilyball · 2019-12-16T23:41:39Z

scripts/create-darwin-volume.sh

+            break
+        fi
+        case "$name" in
+            [Nn]ix*)


Since we're using bash this could be written as

if [[ "$name" == [Nn]ix* ]]; then echo "$name" break fi

lilyball · 2019-12-16T23:44:07Z

scripts/create-darwin-volume.sh

+        echo "  1. Remove the entry from fstab using 'sudo vifs'"
+        echo "  2. Destroy the data volume using 'diskutil apfs deleteVolume'"
+        echo "  3. Delete /etc/synthetic.conf"
+        echo ""


This whole thing could be done as a heredoc

cat >&2 <<EOF ------------------------------------------------------------------ | This installer will create a volume for the nix store and | | configure it to mount at /nix. Follow these steps to uninstall. | ------------------------------------------------------------------ 1. Remove the entry from fstab using 'sudo vifs' 2. Destroy the data volume using 'diskutil apfs deleteVolume' 3. Delete /etc/synthetic.conf EOF

scripts/create-darwin-volume.sh

LnL7 · 2019-12-17T09:14:26Z

@lilyball Given that bash is ancient on macOS I intentionally kept this POSIX sh.

The default login shell for users on macOS 10.15 changed from bash to zsh. So while generally nonstandard we need to configure it to make nix function out of the box on macOS.

This should handle installation scenarios we can handle with anything resembling confidence. Goal is approximating the existing setup--not enforcing a best-practice... Approaches (+ installer-handled, - manual) and configs each covers: + no change needed; /nix OK on boot volume: All pre-Catalina (regardless of T2 or FileVault use) + create new unencrypted volume: Catalina, pre-T2, no FileVault + create new encrypted-at-rest volume: Catalina, pre-T2, FileVault Catalina, T2, no FileVault - require user to pre-create encrypted volume Catalina, T2, FileVault

LnL7 · 2020-05-21T18:04:50Z

The manual and error messages should be much nicer now, uploaded the updated manual here. Since I rebased to fix some merge conflicts it should be tested again but other then that I think it's done.

Thanks @abathur!

abathur · 2020-05-21T21:38:27Z

The manual and error messages should be much nicer now, uploaded the updated manual here. Since I rebased to fix some merge conflicts it should be tested again but other then that I think it's done.

Thanks @abathur!

@LnL7 tested the installer on Catalina + pre-T2 + FileVault and it correctly prompted to pre-create the volume.
I tested Catalina + T2 + (without FileVault & with FileVault). In both cases:
- a bare call to the installer correctly pointed me to the documentation and prompted me to re-run with the --darwin-use-unencrypted-nix-store-volume flag
- re-running with the flag correctly created the volume and installed Nix
I also tested Mojave + pre-T2 + (without FileVault & with FileVault, for good measure) and in both cases it cleanly installed on a bare call to the installer.

scripts/create-darwin-volume.sh

domenkozar · 2020-05-23T09:15:27Z

I'm going to merge this, given that people have been reporting for months that it works (and now that documentation has been written) it seems like not much will improve until we get this live.

ProofOfKeags · 2020-05-24T23:36:40Z

So either I'm missing something or the instructions in the manual definitely don't work. It complains about asking for the flag about --daemon | --no-daemon and I cannot seem to supply either argument. It isn't threading the flags through to the installer correctly. What's going on here?

abathur · 2020-05-24T23:50:30Z

@ProofOfKeags How are you building/obtaining the install script? What exact command are you using?

ProofOfKeags · 2020-05-24T23:51:15Z

$ sh <(curl https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume as stated in the manual link

EDIT: after digging it seems apparent that whatever install script I'm fetching does not even include the option for the --darwin... I gather this hasn't been pushed out to the main host?

abathur · 2020-05-24T23:52:16Z

A new version of Nix hasn't been released yet.

Edit: I don't know exactly what the release/deploy cycle for Nix looks like, but (AFAIK) if you watch the repo for a new release, the command you quoted should work at approximately that time :)

abathur · 2020-05-25T00:01:57Z

Until it's released, my macOS system bootstrap has been doing roughly (I'm adapting it for you--I last ran it before any of this work was merged...) this:

curl https://raw.githubusercontent.com/NixOS/nix/master/scripts/create-darwin-volume.sh | bash
curl https://nixos.org/nix/install | sh

(create-darwin-volume.sh is what the --darwin-use-unencrypted-nix-store-volume flag will cause the main installer to run; it needs to be run before the main install)

abathur · 2020-05-25T00:14:17Z

@ProofOfKeags Aside from this mishap: You may be one of the first to try following the manual update from scratch--any feedback? Do you have questions it didn't answer?

ProofOfKeags · 2020-05-25T00:21:16Z

The guide seemed fairly straight forward, I think. But given that the first command didn't work and I didn't bother trying to go through the rest since it'll work when it is released, I can't say for sure whether the instructions for the rest of it will work. I actually have half a mind right now to partition my mac again and just use NixOS raw, because MacOS seems to be increasingly hostile towards developers.

nathanshelly · 2020-05-25T02:50:36Z

@abathur just for confirmation I was successfully able to remove my single user-install and reinstall a multi-user installation on Catalina + T2 + Filevault earlier today running those same commands

sengaya · 2020-05-29T14:59:55Z

@abathur I can confirm that I could successfully set up nix on Catalina (with T2 chip) with the commands

curl https://raw.githubusercontent.com/NixOS/nix/master/scripts/create-darwin-volume.sh | bash
curl https://nixos.org/nix/install | sh

LnL7 marked this pull request as ready for review November 9, 2019 11:17

domenkozar mentioned this pull request Nov 9, 2019

nix setup failure on macos cachix/install-nix-action#10

Closed

lilyball reviewed Nov 10, 2019

View reviewed changes

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

domenkozar mentioned this pull request Nov 10, 2019

/nix will not be writable on macOS Catalina #2925

Closed

LnL7 force-pushed the darwin-10.15-install branch from 256b22e to c39c829 Compare November 13, 2019 21:31

lilyball reviewed Nov 13, 2019

View reviewed changes

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

garyverhaegen-da mentioned this pull request Dec 10, 2019

document how to get nix on catalina digital-asset/daml#3808

Closed

LnL7 force-pushed the darwin-10.15-install branch from c39c829 to 7543121 Compare December 15, 2019 15:44

LnL7 commented Dec 15, 2019

View reviewed changes

erlend-sh mentioned this pull request Dec 16, 2019

Catalina holochain/docs-pages#149

Closed

matthewbauer reviewed Dec 16, 2019

View reviewed changes

scripts/install-nix-from-closure.sh Outdated Show resolved Hide resolved

matthewbauer reviewed Dec 16, 2019

View reviewed changes

scripts/create-darwin-volume.sh Show resolved Hide resolved

matthewbauer reviewed Dec 16, 2019

View reviewed changes

scripts/create-darwin-volume.sh Outdated Show resolved Hide resolved

lilyball reviewed Dec 16, 2019

View reviewed changes

LnL7 force-pushed the darwin-10.15-install branch from 3832259 to 97c6595 Compare December 17, 2019 09:14

LnL7 and others added 9 commits May 21, 2020 19:58

install: also configure ~/.zshenv

1020262

The default login shell for users on macOS 10.15 changed from bash to zsh. So while generally nonstandard we need to configure it to make nix function out of the box on macOS.

install: show macOS 10.15 message with --daemon

083bb3b

manual: add apfs volume section

ee89b77

install: hide the store volume on darwin

caface1

install: improve output and error handling

04f597c

install: make synthetic.conf and fstab checks stricter

bc24c09

manual: clarify volume creation section

3386575

installer: refuse apfs volume creation when FileVault is enabled

477d7c2

LnL7 force-pushed the darwin-10.15-install branch from f44919c to 2b0a81d Compare May 21, 2020 17:58

installer: don't clobber synthetic.conf

d3df188

sengaya reviewed May 22, 2020

View reviewed changes

scripts/create-darwin-volume.sh Outdated Show resolved Hide resolved

domenkozar reviewed May 23, 2020

View reviewed changes

scripts/create-darwin-volume.sh Outdated Show resolved Hide resolved

scripts/create-darwin-volume.sh: remove unused variable

2a7ea2e

domenkozar merged commit c129e7c into NixOS:master May 23, 2020

toonn mentioned this pull request Mar 14, 2021

Status of macOS NixOS/nixpkgs#116341

Open

ldesgoui mentioned this pull request Aug 1, 2021

Write Nix 2.4 release notes #4521

Closed

erikkrieg mentioned this pull request Sep 17, 2022

Provide a convention for allowing MacOS Spotlight to index Applications without getting hacky #7055

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

install: configure and bootstrap synthetic.conf on darwin #3212

install: configure and bootstrap synthetic.conf on darwin #3212

LnL7 commented Nov 8, 2019 •

edited

Loading

qolii commented Nov 9, 2019

LnL7 commented Nov 10, 2019

lilyball commented Nov 10, 2019

qolii commented Nov 10, 2019

LnL7 commented Nov 10, 2019

domenkozar commented Nov 10, 2019

dhess commented Nov 10, 2019

lilyball commented Nov 11, 2019

mroi commented Nov 11, 2019

arcnmx commented Nov 13, 2019

LnL7 commented Nov 13, 2019

mroi commented Nov 13, 2019

domenkozar commented Nov 18, 2019 •

edited

Loading

LnL7 Dec 15, 2019

matthewbauer commented Dec 16, 2019 •

edited

Loading

lilyball Dec 16, 2019

lilyball Dec 16, 2019

lilyball Dec 16, 2019

lilyball Dec 16, 2019

LnL7 commented Dec 17, 2019

LnL7 commented May 21, 2020

abathur commented May 21, 2020

domenkozar commented May 23, 2020 •

edited

Loading

ProofOfKeags commented May 24, 2020

abathur commented May 24, 2020

ProofOfKeags commented May 24, 2020 •

edited

Loading

abathur commented May 24, 2020 •

edited

Loading

abathur commented May 25, 2020

abathur commented May 25, 2020

ProofOfKeags commented May 25, 2020

nathanshelly commented May 25, 2020

sengaya commented May 29, 2020

install: configure and bootstrap synthetic.conf on darwin #3212

install: configure and bootstrap synthetic.conf on darwin #3212

Conversation

LnL7 commented Nov 8, 2019 • edited Loading

qolii commented Nov 9, 2019

LnL7 commented Nov 10, 2019

lilyball commented Nov 10, 2019

qolii commented Nov 10, 2019

LnL7 commented Nov 10, 2019

domenkozar commented Nov 10, 2019

dhess commented Nov 10, 2019

lilyball commented Nov 11, 2019

mroi commented Nov 11, 2019

arcnmx commented Nov 13, 2019

LnL7 commented Nov 13, 2019

mroi commented Nov 13, 2019

domenkozar commented Nov 18, 2019 • edited Loading

LnL7 Dec 15, 2019

Choose a reason for hiding this comment

matthewbauer commented Dec 16, 2019 • edited Loading

lilyball Dec 16, 2019

Choose a reason for hiding this comment

lilyball Dec 16, 2019

Choose a reason for hiding this comment

lilyball Dec 16, 2019

Choose a reason for hiding this comment

lilyball Dec 16, 2019

Choose a reason for hiding this comment

LnL7 commented Dec 17, 2019

LnL7 commented May 21, 2020

abathur commented May 21, 2020

domenkozar commented May 23, 2020 • edited Loading

ProofOfKeags commented May 24, 2020

abathur commented May 24, 2020

ProofOfKeags commented May 24, 2020 • edited Loading

abathur commented May 24, 2020 • edited Loading

abathur commented May 25, 2020

abathur commented May 25, 2020

ProofOfKeags commented May 25, 2020

nathanshelly commented May 25, 2020

sengaya commented May 29, 2020

LnL7 commented Nov 8, 2019 •

edited

Loading

domenkozar commented Nov 18, 2019 •

edited

Loading

matthewbauer commented Dec 16, 2019 •

edited

Loading

domenkozar commented May 23, 2020 •

edited

Loading

ProofOfKeags commented May 24, 2020 •

edited

Loading

abathur commented May 24, 2020 •

edited

Loading