I'm going to take a risk of being banned for not posting a question.
I have seen tons and tons of questions on how to start an SSH server and it seems to be quite hard, with rebuilding images and what not.
Additionally, being able to connect VSCode to the board directly and remotely to increase development rate is very important.
Below is the ultimate easiest way.
On the board:
Visit and get URL for latest OpenSSH: https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/portable/
wget cdn.openbsd.org/.../openssh-9.3p1.tar.gz
tar -xvzf openssh-9.3p1.tar.gz
cd openssh-9.3p1
./configure --host="arm64"
make
make install
sudo systemctl start sshd.service
sudo systemctl status sshd.service
On your PC or Mac:
Open VSCode and follow these instructions: https://code.visualstudio.com/docs/remote/ssh
IMPORTANT
If VSCode fails to connect and you have previously been able to connect, it may be because the "known_hosts" SSH config file contains an IP with the previous key of the board.
This happens if you e.g reflash your SD.
To fix this, simply open "known_hosts" file and remove the entry with the IP of your board and then reconnect.
The "known_hosts" file can be found here on Windows: C:\Users\<username>\.ssh
Proof that it works:
Final words: Please don't ban me.
God speed!