You can apply the Paragraph attribute of Keep With Next to whichever comes
first, i.e. the caption before the image, apply to the caption. This will
ensure that they will stay together as the document grows, i.e. across pages,
etc.
To move them you will need to select both.
You could have both of them within a borderless two-cell table, but this can
cause problems in the future unless you are very careful.
Hope this helps
DeanH