It is pretty straight forward to migrate an OS X Server's native MySQL 4.1 to a MySQL binary installation of MySQL 5.0. After backing up the databases and /usr/bin install the new binary. Once that is done link /usr/bin/mysql files to /usr/local/mysql/bin.
Two changes are necessary to get everything working. First /etc/my.cnf needs to look at /var/mysql instead of /usr/local/mysql/var for databases. Second since Apple sticks the socket file in /tmp/mysql.sock (come on Apple what are you thinking?) a symbolic link is necessary to have it work properly. The link should point to /var/mysql/mysql.sock (unless you've modified the default config).
The last step is an updatecheck and permissions check. Read more about Upgrading MySQL on Mac OS X Server