How do I left justify the last line only of a fully justified paragraph, in
order to avoid just a few words from being spread across the entire width of
the page?
That is the difference between the "distributed" formatting applied with
Ctrl+Shift+J and the "justified" formatting applied with Ctrl+J. If you select
the paragraph in question and press Ctrl+J, it should behave as you asked.
If that's not it, perhaps the "paragraph" really ends with a manual line break
(Shift+Enter) instead of a paragraph mark (Enter). Turn on nonprinting
characters with the ¶ button and see whether the line ends with the ¶ paragraph
symbol or the left-pointing arrow for a line break.