Yes, a 32-bit architecture is limited to addressing a maximum of 4 gigabytes of memory. Depending on the operating system, this number can be cut down even further due to reserved address space.
This limitation can be removed on certain 32-bit architectures via the use of PAE (Physical Address Extension), but it must be supported by the processor. PAE eanbles the processor to access more than 4 GB of memory, but it does not change the amount of virtual address space available to a single process—each process would still be limited to a maximum of 4 GB of address space.
And yes, theoretically a 64-bit architecture can address 16.8 million terabytes of memory, or 2^64 bytes. But I don’t believe the current popular implementations fully support this; for example, the AMD64 architecture can only address up to 1 terabyte of memory. Additionally, your operating system will also place limitations on the amount of supported, addressable memory. Many versions of Windows (particularly versions designed for home or other non-server use) are arbitrarily limited.