I then attempted to use the card online, at a site that uses PayPal to process customer purchases. My card was denied. In calling AmEx to find out why, I learned the following: the card is only good at sales entities which are direct customers of AmEx. They won't process "third party transactions" such as PayPal. When I asked for my money back, they offered me $15. ($28.95 minus the $3.95 processing fee, minus the $10 refund fee.)
I've decided to keep the card. I'll find somewhere that accepts it (mebbe Starbucks?). However, I'm going to put about $3.95 worth of effort (this post should amount to that) into letting other people know about my experience with AmEx's Gift Card service. I should probably note here that neither their site nor the TOS document that came with the gift card talks about refusing to work with third party services (specifically PayPal). The closest the TOS comes to that is disavowing responsibility if the Merchant declines the card. (In this case, AmEx declined the transaction, not the merchant.)
Bite me, AmEx.