There is no conditional formatting like Excel, but there are a few
alternatives.
First one is to create a highlight filter which you can run.
To do this go to the project menu, selected "filtered for" then more
filters. I think you can probably use one of the late task filters (look
through them to find the one you want. Once you find one you like, instead
of applying the filter copy it and edit it so that the "highlight" box is
checked. The second thing to do is go to format menu, select text styles,
find the "highlight" style and set it the way you want. Now when you run the
filter it will highlight the late tasks using the format you chose.
The second option is to use visual basic to do something similar. Sounds
like you are new to project, so you might want to try this after you have
gotten some more experience.
The third option is to use a custom field with a formula. Choose one of the
unused user defined fields, right click on the column header and select
customize, then click the formula button. You can then write a formula which
determines if a task is late.
If you use a flag field
[Finish]>[Baseline Finish]
should work.
Then in the customize fields dialog box set a graphical indicator. Red if
true and green if false.
It won't change the text formatting but will show a little stoplight or
whatever you prefer.