Most of this is just some recap of already mentioned stuff, partly already rejected. Currently, the book system is kinda working but pretty boring (no, I will not make more interesting suggestions, just a different kind of bore).
The tomes themselves are just some placeholder to slightly restrict healing usage but normally their effect is negligible. Grimoires on the other hand are a brilliant meta-joke about a cursed item, which promises power (better loot) while actually endangering the holder (reducing health) and corrupting its mind (pick up the Grims!!!).
That said, the system has like most systems in any game still potential upwards (debatable if made suggestions are upwards potential).
Off-gameplay potential
It has been suggested several times already by different people to make some kind of library. Aside from a random room in the keep somewhere it could also have some actually usage by a double function. By collecting books the player will unlock more and more game information. Hereby, the Tomes would work to unlock background information on characters, enemies, locations etc. very similar to the Lore pages from Vermintide 1. The information and text for these are already written and would just have to be incorparated into the game. The Grimoires would unlock information about (advanced) gameplay mechanisms. These information would have to be written down first and corrected repeatedly (we could discuss this as a community project, but it would never work out).
Gameplay potential
Tomes are magical books but I am not sure what you could with them. Only idea is something I suggested in the past which would more or less serve as gimmick. As fas as I remember, early versions of Tomes were able to block which was then removed because it was non-sensical. However, I think it would make a tiny bit sense if they could block magic. Which magic would be affected? Basically only Chaos magic: Flies, Leeches and Slime Clones. These could be blocked. Make tome-blocking a thing!!
As for Grimoires, there is far more potential. Although none the community would embrace 
The current problem with Grimoires is that the curse is assymetrical. An IB doesn’t care for having a third of health only as long as he has a passive which allows face-blocking every few seconds. Zealot does care even less because he can face-block about everything and has a passive which literally deactivates death.
As such I would like to make the daring suggestion of personalized curses meaning that every career should have a slightly different curse. The potential ideas here are unlimited. A very easy idea would be for Grimoires to reduce or remove career passives (or even active skills). So if you pick up a Grim then IB (even if he isnt carrier) would have no longer his free hit every few seconds or the cooldown would just take longer. This would mean that the curse would actually hurt and be problematic instead of “I don’t care”. Another idea would be for the curses to deactivate weapon traits.
Or curses are randomized ranging from lower attack speed, to less ammo regeneration, etc. But a curse should be painful for everyone in the team involved. And no, I do not care that this is “not fun”. If people want to pick up a magically cursed artifact which is 100 % optional just to satisfy their greed then they should be prepared for a bit of suffering. And if the result is people picking up grims less then it would be okay as well.
Thematically, debuffs make more sense. But there is of course also the possibility to buff enemies instead. A simple iteration here would be that with every picked up Grim a random Deed modifier would be activated (or similar). So you pick up one Grim, you get continuous horde. You pick up the second grim, now you have more specials as well. Or your heroes will be kidnapped after death It is an option but painful debuffs would make more sense from a lore perspective.