Identifiers (including column names) that are not double-quoted are folded to lowercase in PostgreSQL. Column names that were created with double-quotes and thereby retained uppercase letters (and/or other syntax violations) have to be double-quoted for the rest of their life:
"first_Name"
Values (string literals / constants) are enclosed in single quotes:
'xyz'
So, yes, PostgreSQL column names are case-sensitive (when double-quoted):
SELECT * FROM persons WHERE "first_Name" = 'xyz';
Read the manual on identifiers here.
My standing advice is to use legal, lower-case names exclusively so double-quoting is not needed.