NoMachine requires that the user has already access to his/her home directory when logging-in to the server, in order to be able to create the .nx directory there. If the user cannot access for whatever reason the home directory, the session will fail to start.

This happens for example when the user logs-in for the first time via NoMachine with a AD mobile account: his/her home directory is not already present on the system.

A similar problem occurs also when the AD configuration set users' home directory to /var/empty, which is not accessible by the user.

As a possible workaround, set the 'UsersDirectoryPath' in the node configuration file (node.cfg) to an already existent directory already accessible by the user, for example /tmp/ or /private/tmp.

For NoMachine versions previous to 7.7.4, set  instead the 'UserNXDirectoryPath' key in node.cfg.

On macOS the node configuration file is /Applications/NoMachine.app/Contents/Frameworks/etc/node.cfg, on Linux it's /usr/NX/etc/node.cfg and on Windows C:\Program files (x86)\NoMachine\etc\node.cfg.

As an alternative on macOS you can create the mobile account via command line before the user logs-in by NoMachine:
/System/Library/CoreServices/ManagedClient.app/Contents/Resources/createmobileaccount -n $Username