Windows and doors are in the LCF. To rename, you would have to extract the LCF (which you've done), rename, and then (optionally) create a new LCF. The GUID will remain the same regardless of the name of the file, so this will not affect projects, only the name the user sees.
You do not need to edit the parts to rename - just rename the parts in Mac Finder or Windows Explorer. Yes, all doors/windows will appear in the extracted folders on your disk - you just must not be looking in the right place? If you open the door or window tool in AC, you will see the folder hierarchy of the LCF - the folders created on your disk will match this exactly.
If you mean you want to rename parameters and/or default values, then, yes, you need to edit the parts. This is done via the File > Libraries and Objects > Open Object menu.
Generally, I would advise against renaming - unless you want to keep renaming things with every library update. I'm sure that a patch to the 12 library will be coming before too long ---and you'll then have to repeat your work. And again, and again.
One of the forum moderators AC 27 USA and earlier • macOS Ventura 13.6.2, MacBook Pro M2 Max 12CPU/30GPU cores, 32GB