Not as easily as to get column widths if the height is automatic. The height
is shown only if it's set to an At Least or Exactly setting. You can achieve
this, however, by dragging the bottom boundary of the row just slightly in
Print Layout view. This will cause the row height to be set to an At Least
amount.
Alternatively, with or without moving the boundary, you can press Alt and
click on the boundary, and the height will be shown on the vertical ruler
(it continues to be shown as you drag the boundary). It runs sideways,
though, which requires craning your neck to read it, and it can be difficult
to make out if the cell height is small. Increasing the Zoom can help with
this, though.