Microsoft has a good help page on this: WSL User documentation.
In Windows PowerShell (Super+X, i) you can log in to the Ubuntu root account with
wsl --user root
Then change the password as you usually would, i.e. for the current user (which is root) with
passwd
or for another user with
passwd username
which interactively asks you for a new password (twice). Windows admin privileges are not required.
If you have more than one distribution installed in WSL, then you can list the names of the installed distributions with
wsl -l
and then use the name of the distribution to specify into which distribution you want to log in to, for example to log into Ubuntu 20.04:
wsl -d Ubuntu-20.04 --user root