this post was submitted on 02 Sep 2024
44 points (92.3% liked)
Python
6366 readers
1 users here now
Welcome to the Python community on the programming.dev Lemmy instance!
📅 Events
Past
November 2023
- PyCon Ireland 2023, 11-12th
- PyData Tel Aviv 2023 14th
October 2023
- PyConES Canarias 2023, 6-8th
- DjangoCon US 2023, 16-20th (!django 💬)
July 2023
- PyDelhi Meetup, 2nd
- PyCon Israel, 4-5th
- DFW Pythoneers, 6th
- Django Girls Abraka, 6-7th
- SciPy 2023 10-16th, Austin
- IndyPy, 11th
- Leipzig Python User Group, 11th
- Austin Python, 12th
- EuroPython 2023, 17-23rd
- Austin Python: Evening of Coding, 18th
- PyHEP.dev 2023 - "Python in HEP" Developer's Workshop, 25th
August 2023
- PyLadies Dublin, 15th
- EuroSciPy 2023, 14-18th
September 2023
- PyData Amsterdam, 14-16th
- PyCon UK, 22nd - 25th
🐍 Python project:
- Python
- Documentation
- News & Blog
- Python Planet blog aggregator
💓 Python Community:
- #python IRC for general questions
- #python-dev IRC for CPython developers
- PySlackers Slack channel
- Python Discord server
- Python Weekly newsletters
- Mailing lists
- Forum
✨ Python Ecosystem:
🌌 Fediverse
Communities
- #python on Mastodon
- c/django on programming.dev
- c/pythorhead on lemmy.dbzer0.com
Projects
- Pythörhead: a Python library for interacting with Lemmy
- Plemmy: a Python package for accessing the Lemmy API
- pylemmy pylemmy enables simple access to Lemmy's API with Python
- mastodon.py, a Python wrapper for the Mastodon API
Feeds
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I started to use Nix to build containers that contain just my app and nothing else. The benefit of it is that it makes containers smaller, removes unused components (less potential attack vectors) and a container from a specific checked out version will always be identical (Dockerfile on its own (without extra work) doesn't provide such guarantee). I also have the ability to customize python and dependencies to remove additional pieces that I don't need (this unfortunately requires some experience with Nix, to know how to do it)
I wrote my own abstraction on top of poetry2nix and nix2container to remove need for boilerplate: https://github.com/takeda/nix-cde
The example shows how a hello world application can be packed and then how I can reduce its size further from 178MB to 68.9MB. This doesn't include using musl to get the size even lower than that.
Though I totally agree with author about venv and that's what I did before and still do in situations where I can't use Nix. Venv is standardized and is much more predictable and prevents surprises.
That's true, but also misleading.
OCI image is like having an jpeg image. While Dockerfile is like the text prompt you write to ChatGPT to generate the image.
Yes every time you look at the jpeg, it is the same exact image, but that's kind of obvious, the real problem is if you try the text query to ChatGPT you will get something slightly different every time.
Nix brings a true reproducibility. So in this analogy the same prompt brings the exact same image. This allows you to check on that prompt in your source control and if you mess up something there's always a way back.
This is something docker promised, but never delivered.
It should not, but artifacts never had problem with mutating before we had docker. If you generate an rpm package and store it in an artifactory it always was the same exact package (unless someone overwrote it, lol)
But that's basically the problem docker claimed to fix. This is also the problem that you frequently encounter with a pipeline that worked fine one day suddenly stopped working next day, because something that your Dockerfile referenced changed (maybe a new image was updated that broke something, you can lock things to specific hashes, but you need to be very conscious about that and in the wild I never seen anyone really doing it).
It is not. Hashes are and lock files are built-in and Nix uses them by default.
If for example I use a flake, the flake.lock will hold the exact version of nixpkgs (package repo) in time. That happens without any additional effort. The poetry2nix converts poetry.lock file to nix packages that are once again locked in time, and that also happens behind the scenes.
The result is that all dependencies (python dependencies - from poetry.lock as well as the rest of the system (python, c libraries etc) - from flake.lock are all locked and in my repo. So everything is repeatable without effort on my side.
To repeat that with Dockerfile is much more challenging.
If you get your app build with Nix. The whole thing, including all of app's dependencies are explicitly referenced so you can wrap it into a docker, an rpm file, OS image etc.
It's controversial, but IMO nix is actually easier than what we are doing now. I think the problem is that it is a massive paradigm shift and what most people know what to do with existing technologies will generally be not useful, so you have to relearn everything.
But IMO it pays off. For example when starting a new project I can package the whole thing in 5 minutes. poetry2nix translates the project and it's dependencies into nix packages and then since nix understands dependencies for my project it can package it automatically.
You make a good point in that Docker promised to make dev environments reproducible so that everyone on the team would have the same environment. They even succeeded in that, but either intentionally or accidentally omitted reproducibility over time due to the introduction of non-locked dependencies.