I have tried finetuning extensively, a typical run over a pre-trained set before expanding the number of classes has the loss steadily increasing without any clear indication of how long that would last. Maybe I should let a test run for a couple of days to see if it will eventually converge.
Also, keep in mind that the dataset is still tiny and that a method that works for large numbers of images may very well fail if you only have a few tens to maybe 100 or so images per class.
For finetuning on additional data, you would have to lower the learning rate because you're only adding a few datapoints and it's almost entirely converged as it is. If your loss is increasing, that suggests overfitting to me via a too-high learning rate
Now, if you're changing the architecture (such as by adding additional categories of pieces), as I said, that's more tricky - what people usually do there is something like lop off the top layers and retrain them from scratch, possibly while freezing the rest of the NN (the assumption there being that the learned filters and lower layers ought to already be sufficient to classify a new category, which is reasonable since the lower layers tend to be learning things like lines and corners, all primitives which should be able to classify yet another square or rectangle etc).
Since this is the obvious response any reader familiar with deep learning would have while reading complaints about how slow your CNN is to train from scratch, it'd be good to discuss it in some detail what sort of finetuning you've tried and how it failed.
I will send you an email with a re-run of my original experiments, they were roughly what you described (take a pre-trained net, remove the last layer and re-connect to a layer with the right number of classes), learning rates I tried were from 1e-6 to 1e-3 and none of those had satisfactory results.
I was about ready to give up on it when I decided to try to bring up a net from scratch and that worked quite well.
Also, keep in mind that the dataset is still tiny and that a method that works for large numbers of images may very well fail if you only have a few tens to maybe 100 or so images per class.