2 things:
1) pen sets are super unstable in the layout book
2) the pen set and layout combination should not be freely editable by simply opening pen set or layer manager with a layout open. this is a goofy user interface at best. at worst, it is problematic for teamwork projects, since nothing is required to be reserved to edit the layoutbooks pens or layer combo. These layout book settings SHOULD be located in the layout book settings, or in the subset settings, or even in the individual layout settings. But not be freely edited as a global adjustment, and certainly not editable without requiring a reserve (something) first.
BIM solutions and trouble shooting (self proclaimed) expert. Using Archicad 26 5002 US on Mac OS 11.5.2