Okay, I got it worked out. There were a few things going on.
First, the com.ibm.xsp.extlib.util.ExtLibUtil and com.ibm.xsp.extlib.util.JdbcUtil classes are in the relational database plugin of the XPages Extension Library. That's why I couldn't find them.
Second, when I first copied the sample database to my server it gave me some permission errors when I tried to access the pages. So I signed the database. Then some of the pages seemed to work, others just gave errors about the DBUtil class, etc. That's when I looked at the code and saw the compilation errors related to ExtLibUtil and JdbcUtil.
Third, to fix all of this I needed to install the Extension Library plugins into DDE, restart DDE, and do a full Build on the database. Then everything worked (I already had the Extension Library installed on my server). All good now.
It might be good to add something in the readme file about the ExtLib dependencies, if they're not there already. Or, if I totally missed that part of the instructions, good job and please just forget I said anything... :-)