Local AI RAG with LLama-3 llama.cpp

ChatGPT is best chat bot for now, IMHO… It’s ok to explore themes, philosophize... It’s not so heavily censored, it imitates logical thinking process very good. But at the same time it lies sometimes – distorts names of books, gives bad links, leads you astray with technical staff, like electronics and administrating… It’s nice to argument with it but when getting tech solutions and advice one should be cautious. Also it loves throwing rhetorical garbage.

Gemini. Very helpful in technical staff – short and precise. Good in programming. Worse with electronics than ChatGPT. Didn’t chat a lot with it, so do not know how "well spoken" it is, but generally feels neutral avoiding bold conclusions. Great addition to conventional Google Search.

Grok. Have mixed feeling of it. Very helpful with system administrating. In programming sometimes better than ChatGPT, but as a chat bot it is censored and avoids controversial topics. Even when cornered logically, dodges arguments to stay tolerant, which is bad.

Perplexity. Brief and straight to the point. Didn't used it much.

All local LLM models, I tried, suck… But maybe large ones for very powerful GPUs are OK, though I have no inclination to invest in hardware right now.

We can't rely on AI opinions, but as support and augmentation to idea exploration AIs are OK.

VAG KKL 409.1 on Ubuntu Linux

sudo apt install wine // and install VCDS, cable drivers already are in Linux core, interface will be found and cable indicator will be active only when connected to car, on PC - Port OK, interface not found.

Identify COM ports on Linux

sudo dmesg | grep tty

ls /dev/ttyS* // typically refers to hardware serial ports.

ls /dev/ttyUSB* // refers to USB-to-serial converters.

ls /dev/ttyACM* // refers to USB CDC (Communications Device Class) devices.

lsusb

udevadm info -e | grep -E 'DEVNAME|ID_SERIAL'

Create the Symbolic Link

cd ~/.wine/dosdevices

ln -s /dev/ttyUSB0 com1 // map /dev/ttyUSB0 to COM1

rm com1 // if it exists and not needed

ls -l ~/.wine/dosdevices // verify the symbolic link

Add user to dialout group

sudo usermod -aG dialout $USER // add your user to the dialout group

cat /etc/group

grep dialout /etc/group

groups user

Change permissions

ls -l /dev/ttyUSB0 // list permissions

crw-rw---- 1 root dialout 188, 0 May 28 12:06 /dev/ttyUSB0

sudo chmod a+rw /dev/ttyUSB0 // change permissions

sudo lsof /dev/ttyS0 // If no output is returned, no process is currently using it. Ignore error.

Create a Test Script

sudo apt-get install python3-serial

touch test_serial.py

import serial

port = '/dev/ttyS0'
baudrate = 9600

ser = serial.Serial(port, baudrate, timeout=1)
ser.write(b'Hello, serial port!\n')
response = ser.readline()
print(response.decode('utf-8'))

ser.close()

python3 test_serial.py // run script

Install Python3.11 for Stable Diffusion after Python.310

If you upgraded to python3.11* on Linux => No module named pip error. 

Launching launch.py...
################################################################
Using TCMalloc: libtcmalloc_minimal.so.4
Python 3.11.4 (main, Jun  9 2023, 07:59:55) [GCC 12.3.0]
Version: v1.5.1
Commit hash: 68f336bd994bed5442ad95bad6b6ad5564a5409a
Installing torch and torchvision
/home/user/Downloads/stable-diffusion-webui/venv/bin/python3: No module named pip
Traceback (most recent call last):
 File "/home/user/Downloads/stable-diffusion-webui/launch.py", line 39, in <module>
   main()
 File "/home/user/Downloads/stable-diffusion-webui/launch.py", line 30, in main
   prepare_environment()
 File "/home/user/Downloads/stable-diffusion-webui/modules/launch_utils.py", line 311, in prepare_environment
   run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True
)
 File "/home/user/Downloads/stable-diffusion-webui/modules/launch_utils.py", line 113, in run
   raise RuntimeError("\n".join(error_bits))
RuntimeError: Couldn't install torch.
Command: "/home/user/Downloads/stable-diffusion-webui/venv/bin/python3" -m pip install torch==2.0.1 torchvision==
0.15.2 --extra-index-url https://download.pytorch.org/whl/cu118
Error code: 1

sudo apt install python3-venv python3-pip

Delete venv folder in Stable Diffusion installation:

/home/user/Downloads/stable-diffusion-webui/venv/

./webui.sh --xformers

Stable Diffusion will install everything, it needs into new venv folder.

On Windows no need to do this. Use python3.10.6.

https://bobbyhadz.com/blog/python-no-module-named-pip

Adding Python deadsnakes ppa repository on Kubuntu Linux

apt-cache policy python3

sudo add-apt-repository ppa:deadsnakes/ppa

https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa

MAKE SURE your Linux version is supported: https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu/dists/

IF NOT, use latest supported version for ppa

sudo nano /etc/apt/sources.list.d/deadsnakes-ubuntu-ppa-jammy.list

deb https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu/ focal main
deb-src https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu/ focal main

sudo apt edit-sources // edit repositories

sudo nano /etc/apt/sources.list // edit repositories

ls /etc/apt/sources.list.d // list additional repositories files

/etc/apt/sources.list.d$ sudo dpkg --force-depends --purge libpython3.10-stdlib
dpkg: libpython3.10-stdlib:amd64: dependency problems, but removing anyway as you requested:
python3.10 depends on libpython3.10-stdlib (= 3.10.7-1ubuntu0.4).

(Reading database ... 254967 files and directories currently installed.)
Removing libpython3.10-stdlib:amd64 (3.10.7-1ubuntu0.4) ...
/etc/apt/sources.list.d$ sudo dpkg --force-depends --purge libpython3.10
dpkg: warning: ignoring request to remove libpython3.10 which isn't installed
/etc/apt/sources.list.d$ sudo dpkg --force-depends --purge libpython3.10-minimal
dpkg: libpython3.10-minimal:amd64: dependency problems, but removing anyway as you requested:
python3.10-minimal depends on libpython3.10-minimal (= 3.10.7-1ubuntu0.4).

(Reading database ... 254625 files and directories currently installed.)
Removing libpython3.10-minimal:amd64 (3.10.7-1ubuntu0.4) ...
Purging configuration files for libpython3.10-minimal:amd64 (3.10.7-1ubuntu0.4) ...
/etc/apt/sources.list.d$ sudo dpkg --force-depends --purge libpython3.10
dpkg: warning: ignoring request to remove libpython3.10 which isn't installed

/etc/apt/sources.list.d$ sudo apt install python3.10
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
python3.10 : Depends: python3.10-minimal (= 3.10.12-1+focal1) but 3.10.7-1ubuntu0.4 is to be installed
             Depends: libpython3.10-stdlib (= 3.10.12-1+focal1) but it is not going to be installed
python3.10-minimal : Depends: libpython3.10-minimal (= 3.10.7-1ubuntu0.4) but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

sudo apt purge python3.10

sudo apt purge python3.10-minimal

sudo aptitude

https://packages.ubuntu.com/kinetic/libpython3.10-stdlib

sudo dpkg -i libmpdec3_2.5.1-2build2_amd64.deb

sudo dpkg -i libpython3.10-minimal_3.10.7-1ubuntu0.4_amd64.deb

sudo dpkg -i libpython3.10-stdlib_3.10.7-1ubuntu0.4_amd64.deb

sudo apt --fix-broken install

Spent lot't of time and failed...

Change the Python3 default version in Ubuntu

cd /usr/bin

sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1

sudo update-alternatives --config python // choose python version

python3 --version

Python 3.10.7

sudo apt-get install --reinstall ca-certificates

See Technical details about this PPA => Signing key: => Fingerprint:

https://askubuntu.com/questions/1459362/how-to-resolve-the-key-not-available-error-on-deadsnakes-ppa

https://www.rosehosting.com/blog/how-to-install-and-switch-python-versions-on-ubuntu-20-04/

https://unix.stackexchange.com/questions/410579/change-the-python3-default-version-in-ubuntu

https://askubuntu.com/questions/1239829/modulenotfounderror-no-module-named-distutils-util

Remove python3 - Python 3.11 on Ubuntu

ls /usr/bin/python* // list all installed python versions

[Bug]: RuntimeError: Couldn't Install Torch

Stable Diffusion----------------------------------------------------------------------------

./webui.sh --xformers

################################################################

Install script for stable-diffusion + Web UI

Tested on Debian 11 (Bullseye)

################################################################

################################################################

Running on user user

################################################################

################################################################

Repo already cloned, using it as install directory

################################################################

################################################################

Create and activate python venv

################################################################

################################################################

Launching launch.py...

################################################################

Using TCMalloc: libtcmalloc_minimal.so.4

Python 3.11.4 (main, Jun  9 2023, 07:59:55) [GCC 12.3.0]

Version: v1.5.1

Commit hash: 68f336bd994bed5442ad95bad6b6ad5564a5409a

Installing torch and torchvision

/home/user/Downloads/stable-diffusion-webui/venv/bin/python3: No module named pip

Traceback (most recent call last):

  File "/home/user/Downloads/stable-diffusion-webui/launch.py", line 39, in <module>

    main()

  File "/home/user/Downloads/stable-diffusion-webui/launch.py", line 30, in main

    prepare_environment()

  File "/home/user/Downloads/stable-diffusion-webui/modules/launch_utils.py", line 311, in prepare_environment

    run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True)

  File "/home/user/Downloads/stable-diffusion-webui/modules/launch_utils.py", line 113, in run

    raise RuntimeError("\n".join(error_bits))

RuntimeError: Couldn't install torch.

Command: "/home/user/Downloads/stable-diffusion-webui/venv/bin/python3" -m pip install torch==2.0.1 torchvision==0.15.2 --extra-index-url https://download.pytorch.org/whl/cu118

Error code: 1

----------------------------------------------------------------------------Stable Diffusion

sudo rm /usr/bin/python3.11

sudo apt purge python3.11

usr/bin$ sudo apt remove python3.11-minimal

sudo find / -xdev -name python3.11*

sudo find / -xdev -name 'python3.11*' -exec rm -r {} \; // remove all found

sudo apt-get purge python3-uno

sudo apt-get install python3-uno

sudo apt-get purge python3.11-minimal python3.11 python3 python3-uno

sudo apt update -y && sudo apt upgrade -y

sudo apt --fix-broken install

sudo apt autoremove --purge *python* // remove all python packages

apt-cache policy python3

SOLUTION in my case:

sudo dpkg --force-depends --purge libpython3.11-stdlib

sudo dpkg --force-depends --purge libpython3.11

sudo dpkg --force-depends --purge libpython3.11-minimal

sudo apt --fix-broken install

AVOID creating conflicts by installing different kernel version updates!!!

sudo apt install -f --reinstall python3-minimal

sudo apt install -f --reinstall python3-pip

sudo apt install -f --reinstall python3

sudo apt update

sudo apt upgrade

sudo apt full-upgrade

sudo apt reinstall plasma-desktop

sudo apt reinstall kwin-x11

https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-programming-environment-on-ubuntu-22-04

https://askubuntu.com/questions/1402419/cant-remove-python-3-10-from-ubuntu-20-04lts

https://gist.github.com/zhensongren/811dcf2471f663ed3148a272f1faa957

https://computingforgeeks.com/how-to-install-python-on-ubuntu-linux-system/?expand_article=1

https://linuxhint.com/uninstall-python-ubuntu-22-04/

https://www.debugpoint.com/install-python-3-10-ubuntu/