Website generation on multiple core/cpu

[SBMOffshore]

hello,

Is there any way to generate website on multiplie core/cpu in order to improvement generation time?

So far only one core is mobilized and it can significantly improve the generation speed if the remaining cores are used.

thank you,

[jgronau]

Bonjour Jerome,

are there any Changes to this Facts ? For us it is a Problem which we want to fix.

Cordialement Jens

[jgronau]

Bonjour Jerome,

are there any Changes to this Facts ? For us it is a Problem which we want to fix.

Cordialement Jens

[jhorber]

Hello K6i6

In the mechanism described in the above KB (http://community.mega.com/t5/custom/page/page-id/m​ega-kb-solution?sid=50120000000mqmkAAA)

• The first phase (to compute pages to be generated) is run from the beginning
• The second phase is resumed from the last page generated (saved in a .ini file)

This mechanism does not rely of a persistent identification of 'new objects'

In general, if you need to identify objects that have changed of that are new, you can use the repository log (at repository level, not the private workspace log). This data material is used by features such as alert management or full search.

You could consider to update dynamically (with API script) a web site to control the list of objects to document (from the list identified in repository log)

However,you cannot control the scope of objects used to compute hyperlinks and indexes at generation

The hyperlinks and indexes would considers the 'complete scope' (previous list of objects + additional list of objects), only the additional scope.

[ki6i]

I meant delta in sense that it should generate only the objects that were modified since the last generation, because otherwise we endup generating every day 50 000+ pages and most often only 1000-2000 of these objects were actually updated.

Unfortunatelly there is no way to identify this objects in mega because the Modification Date is updated only on Attribute change and not on Association connect/disconnect change. Furthermore, the reports are clickable only if all the objects that are part of them are part of the scope of the generation.

[jhorber]

Hello Ki6i

As far as I know, there is no project to enable static web site generation with multiple threads or multiple cores.

The current bottleneck is the limit of a windows 32 process around 2.5 GB ram with recent operating systems.

An error handling mechanism exists. Consider to use it for incremental generation

See http://community.mega.com/t5/custom/page/page-id/mega-kb-solution?sid=50120000000mqmkAAA

[ki6i]

If we can request features I would really like if the website generation is actually updated to handle at least couple of threads, because that will decrease the generation time tremendously. Especially considering that we can’t make delta generation.

[jhorber]

Hello

HOPEX does not enable this kind (parallel processing) of static web site generation

To reduce generation time, consider various indications

Deployment

• Increase CPU frequency/ram of the machine where generation runs
• Improve performances of network connection between machine where generation runs and database server (for RDBMS storage)
• Avoid to cross network to save html pages (set target generation folder on local disk)

Web site design

• Review scope (maybe there is not need to generate certain pages)
• Consider to user ERQL sets. See KB http://community.mega.com/t5/custom/page/page-id/mega-kb-solution?sid=501D0000000N59LIAS