I've known since I was a kid that a sawbuck was a $10 bill and a fin was a $5. But I had no idea how limited that usage is or even how it came about. I had heard about the Spanish doubloon being the basis for the American dollar and also being divisible into 8 pieces - hence the term "pieces of 8" in pirates' tales and songs. Two of these pieces would be a quarter doubloon or quarter dollar. That's where we get 25 cents being 2 bits. Four bits was also heard in my childhood but I haven't heard it lately.
When Carolyn and I were dining at the Buckhorn Grill, there was a little sign on our table (and all the others as well) advertising sawbuck wine. I expected it to be $10 wine but in fact a glass costs $6.95 and a bottle $20. Seems to me that someone is a little unclear on the concept of sawbuck. Sort of like Jack in the Box's $6 hamburger which sells for more or less but never exactly $6.
Anyway, the best I could find on the Internet was that a sawbuck was either a rack for cutting rough wood, as illustrated above, or the Railroad Crossing sign. Both have a big Roman numeral X, signifying 10. A $10 bill would thus be related. It's a pretty tenuous connection.
For a $5 bill, the connection is finf, Yiddish for 5. Thus, to call a $5 bill a fin is just a minor corruption of yiddish.
There doesn't appear to be the same consistency for naming a $20 bill or higher but back when sawbuck and fin were common, $20 bills weren't.