I found that captions that are set with right alignment actually float to the left. This issue exists in both Tower and Specular.

Adding the following fixes the problem, should you wish to add this in an upcoming release for both themes:

.wp-caption.alignright {
float: right;
}