xkcd #1987: Python Environment

It feels like you stood behind me yesterday, taking notes.

This comment gives me ptsd

thats when you do

/usr/bin/python3.11 -m pip install

mother. fucking. hardcoded paths. 1 step forward, 10 steps backward.

I recommend distrobox for adhoc distrohopping. Though for Nvidia stuff it links to drivers and cuda that you have installed on your host, so... I recently needed cuda 11.8 and that was hella fun to get going.

This is useless. Why don't you start a community for google searches?

Exactly, venv solves this issues for me

I've been using pipenv for a good while but I've started to move over to venv slowly, and I like it so far. It's a bit more of manual work but I feel like it's worth it.

Seconded, after being burned repeatedly I always do this. But why are you calling activate from the directory above?

I started using poetry on a research project and was blown away at how good it is. Next week I start a new job and I am hoping it is the standard.

Yessssss

I actually wrote a script to make a folder an instant pipenv environment for me. Add it to your ./.zshrc. Has saved me a ton of time, I just removed some spaghetti lines that would reinstall pip and shit because it's when I was still early days into Py dev, now I work more with Py than I do C# and I'm a senior C# engineer, I just enjoy the masochism of py.

Also added a check for Arch/Ubu.

# Automated python virtual environment.
#######################################
VENV(){
if ! [ -x "$(command -v pipenv)" ]; then
     echo "pipenv not installed... installing it now!"
     sudo pip install pipenv
     OS="$( ( lsb_release -ds || cat /etc/*release || uname -om ) 2>/dev/null | head -n1 )"
     if [[ "$OS" == *"buntu"* ]]; then
        sudo apt install pipenv -y
     elif  [[ "$OS" == *"rch"* ]];  then
        sudo pacman -S pipenv
     fi
     pip install pipenv --upgrade
     echo "Installation complete!"
fi
if [ -n "$1" ]; then
        echo -e "Args detected, specifically using version $1 of python in this project!"
        version="$1"
else
        version=$(python -V)
        version=$(echo "$version" | sed -e 's/Python //')
        if [ -z "$version" ]; then
                version=$(python3 -V)
                if [ -z "$version" ]; then
                         echo "No python version installed... exiting."
                         return
                fi
        fi
fi
echo -e "\n===========\nCreate a Python $version virtual environment in $PWD/.venv [y/n]?\n==========="
read -r answer
case $answer in
    [yY][eE][sS]|[yY])
export PIPENV_VENV_IN_PROJECT=1
pipenv --python "$version"
pipenv install -r ./requirements.txt
echo -e "\n\n\nVirtual python environment successfully created @ $PWD/.venv!\n"
echo -e "To run commands from this dir use 'pipenv run python ./main.py'"
echo -e "To enter a shell in this venv use 'pipenv shell'."
echo -e "To install from a requirements text file use 'pipenv install -r requirements.txt'"
echo -e "To update pip + all pip modules use 'pipenv update'!\n"
echo -e "Additional information can be found @ https://pipenv-fork.readthedocs.io/en/latest/basics.html"
;;
    [nN][oO]|[nN])
        echo "Fine then weirdo why did you run the command then, jeez.Exiting"
;;
 *)
 echo "Invalid input..."
 ;;
 esac
}

I could redraw this whole chart using only references to pipenv based on my experiences with managing it alongside other tools (especially homebrew). It’s good at many things but is no magic bullet.

I've been burned by pipenv before on a large project where it was taking upwards of 20 minutes to lock dependencies. I think these days they use poetry instead, but I've heard the performance is still not very scalable

With that said, I think it can be a nice addition, but I think it comes down to Python packages not really taking dependency management as a top priority instead of favoring flexibility. This forces a package manager to download and execute the packages to get all the dependency information. Naturally, this is a time-consuming process if the number of packages is large.

On multiple instances I've seen projects abandon it for pip and a requirements.txt because it became unmanageable. It's left a bad taste in my mouth. I don't like solutions that claim to solve problems but introduce new ones.

Do you have any troubles running it as your daily OS? Do you use it as your hobby or also for your work?

I know Nix and use it as my package manager, but I'm not sure about the experience with NixOS. So I'm still reluctant to make the switch.

I agree. I use a Mac but use nix to manage all this mess.

And the vscode direnv extension just makes it all work together.

Wait, you guys don't have a vm for each project and just use ssh to work on them?

I've been trying to sell this idea to my team for a year now. I've even done all the legwork in my free time with a personal project and I've offered the patterns to the team. But alas, we still commit to masochism.

Agreed! That's the way we do work projects. For personal stuff, I also like using pyenv. But yeah, that's it, keep it nice and simple.