yboutros

Meaning, VMs with Xen and hardware virtualization support

The system VM/Qube for USBs is isolated, the Network VM/Qube is separate and isolated, the windowing system and OS housing the qubes is isolated....

And being able to configure all of those with Nix would be a wet dream come true

here's my nix config for hacking that I stripped from my general purpose configuration.nix

I'd like to add whatever other tools other professionals find useful for staying anonymous, reconning, fuzzing, reverse engineering... anything part of the work flow

{ config, pkgs, inputs,lib,  ... }: 
{
  # Define necessary services
  services = {
    tor = {
      enable = true;
      client.dns.enable = true;
      settings.DNSPort = [{
        addr = "127.0.0.1";
        port = 53;
      }];
    };
    resolved = {
      #enable = true; # For caching DNS requests
      fallbackDns = [ "" ]; # Overwrite compiled-in fallback DNS servers.
    };
  };
  # set hist size to 0 or 5
  # https://nixos.wiki/wiki/Encrypted_DNS
  networking.nameservers = [ "127.0.0.1" "::1" ];
  networking.dhcpcd.extraConfig = "nohook resolv.conf";
  networking.networkmanager.dns = "none";
  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.yboutros = {
    isNormalUser = true;
    description = "ynotboutros";
    extraGroups = [ "networkmanager" "wheel" "audio" "kvm" "adbusers"];
    packages = with pkgs; [
      shred 
      gdb # Debugging binaries
      proxychains # Tunneling through proxies such as tor
      tor # Dark Net
      tor-browser # Dark Net
      signal-desktop # Comms
      bleachbit # clean up after yourself
      qdirstat # File statistics
      tree
      awscli2 
      awsls
      python311
      librewolf # Clearnet browser
      electrum # Bitcoin wallet
      nmap # Recon for identifying ports to services
      dig # Provides records and IPs associated with a hostname
      wireshark # Network Analysis tool
      zap # Web app vulnerability scanner
      metasploit # General framework
      bisq-desktop # Decentralized marketplace for buying crypto
      element-desktop-wayland # Matrix element comms app
      veracrypt # Creates encrypted volumes
    ];
  };

  environment.systemPackages = with pkgs; [
   ascii # Useful encoding tools
   busybox # Useful unix tools
   android-tools # ADB, etc...
   qFlipper # SDR: Controlling Flipper Zero
   gnupg # Encryption tools
   torctl # routes all traffic through tor with torctl.sh start
  ];
}

Went through the pain of packaging a python project on Nixos. Here's some issues I hit, and how I got lucky resolving them. I feel the most reliable way of doing this in the future is to use docker and just imperatively build.

Here's how I got web drivers, AI dependencies, gpu dependencies, and an api dependency bundled together into an ephemeral shell for python development, on NixOS 23.11

0. Enable Flakes

1. Start with setting up poetry2nix

2. Get the template flake by running nix flake init --template github:nix-community/poetry2nix

3. in the flake.nix, sometimes changing projectDir = self to projectDir = ./. fixed some issues

4. in your terminal, run nix develop . to build the poetry app with python packages described in pyproject.toml

5. By default, just poetry and python latest should be installed. the dependencies for the project (which gets reflected in the pyproject.toml) are updated with poetry add, such as poetry add numpy selenium scikit-learn

6. Exit out of the ephemeral shell from nix develop ., and rerun to have poetry2nix rebuild and link the newly declared packages

Poetry2nix has worked pretty well for the more obscure python packages, but failed in others. For example, sentence-transformers would depend on maturin, which would fail to link setuptools. If poetry doesn't work, you can try and get the package from nixpkgs, or specify sha256s from pypi.org

Here's an example of what I added to my flake.nix to get gpu acceleration, sentence-transfomers, firefox drivers for selenium, and other packages poetry failed to setup:

packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
];

was added to this flake.nix, as in,

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:nix-community/poetry2nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        pkgs = nixpkgs.legacyPackages.${system};
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication;
      in
      {
        packages = {
          myapp = mkPoetryApplication {
            projectDir = ./.;
          };
          default = self.packages.${system}.myapp;
        };
        devShells.default = pkgs.mkShell {
          inputsFrom = [ self.packages.${system}.myapp ];
          packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
          ];
          nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
          )];
        };
      });
}

There was one package (serpapi), which was not in nixpkgs, and poetry failed as well. Adding this to native build inputs got serpapi installed

nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
)];

All in all, it works, and I have no doubt I've made a reproducible environment. What attracts me is I've never had an easier time setting up cuda/cudnn/tensorrt/... system drivers have been near effortless, and much faster to setup than on debian. Tools like sentence-transformers and torch default to packages which leverage the GPU.

What pushes me away, is I've had failures in each of the three methods for specifying package dependencies, even though one of the three eventually was the fix for integrating the dependencies into my shell. For now, I'll stick with it, but it's hard for me to suggest to a team we use this in development

I setup a next.js project with pkgs.mkshell, and used nix develop to automatically build the project. However, when I leave the shell, the files persist. How should/can(?) I setup my shell.nix so that files in the directory it drops down into are automatically removed when leaving the ephemeral shell?