repair metadata takes too much time

Description

Processing command: 'repair metadata search of type mods' in 2019.06 only needs 4 minutes for 5.000 documents. Using 2020.06 with ~3.000 documents needs 36 minutes for the same command.

Log 2019 for one single repair step:

Log 2020 for one single repair step:

Environment

None

Activity

Show:
Kathleen Neumann
August 20, 2020, 9:00 AM

I think we are loosing the time here:

That doesn't look like much, but in sum it is clearly noticeable. My calculation said, that we need 0,72 sec for repairing one document (was 0,048 in 2019.06). Repairing 120.000 documents need now one day (was ~2 hours before).

Kathleen Neumann
August 20, 2020, 9:40 AM

seems to be related to MCR-2113

Kathleen Neumann
August 24, 2020, 7:14 PM
Edited

After some profiling MCREntityResolver seems to use most of the time, see screenshot attached. Changes in this class where made in MCR-2130.

Sebastian Hofmann
August 25, 2020, 2:28 PM

The slow repair is caused by the static content generator: see ifs_handler_*.png screenshot.

Assignee

Sebastian Hofmann

Reporter

Kathleen Neumann

Labels

None

URL

None

External issue ID

None

Fix versions

Affects versions

Priority

Medium
Configure