Hi

I created a website with CoucCMS that we use as a backend, processing hundreds of requests per day. A frontent website sends POST-Requests to a template in the backend website, that processes this data, creates a new data page of a clonable template, sends a mail to the cusromer and then redirects back to a thank you page of the originating website.

In the backend website, some pages take some seconds to load, because it generates lists with thousands of instances of cloned pages with about a dozen of fields. Some other pages generate statistical charts that take also amstutz gnoficant amount of time. These pages are already highly optimized with dedicated SQL queries, but sometimes the creation of a statistics page still needs about a minute.

Now to the problem: When an internal user (employee) loads such a time consuming page and, while the page is loading, a customer sends a request from a frontend website, it is blocked until the other page is loaded, and only after that it is redirected. So the customers waits up to about a minute in the worst case, without gett ng a feedback.

Does CouchCMS block all other page requests when processing a database request? Or can I change some settings so that the pages do not have to wait for another page finished loading?

I already tried running the API-template page without accessing to the database (no get_field, pages, etc.), but it is still blocked.

Any idea?

Regards,
Olliwalli

[trendoman]

Oh, this is not new, I've seen it and reported to @KK a while ago. From my local experiments server's load (your thousands of pages) doesn't matter. It is a known PHP Session Locking (google it) which happens in Couch. My advice 1: avoid lengthy operations during page request completely by setting up jobs in background in separate delayed request & 2: go through Couch code and amend code to deblock session.

[trendoman]

Maybe it is possible in your application to have 2 installations of Couch - one for frontend and one for analytical things, sharing the same database. Different PHP scripts therefore should not block each other's requests in theory. Interesting to know if you gonna try any of the possible solutions.

I found some code I added a while ago that slowed down the application, or stopping cache from working respectively. After removing this the app loads faster, and after reloads quickly after initial load it seems that the results of the requests done are cached.

It is the following code which served for proper trimmed output of the page:

Code: Select all

<?php ob_start(); require_once( 'couch/cms.php' ); ?>
...
...
...
<?php COUCH::invoke( K_IGNORE_CONTEXT ); ?>
<?php
   $content=ob_get_clean();
   echo trim($content);
?>

BUT, the problem of blocking parallel loading of pages/requests is not solved.

Hi everybody

I am working on the problem again. What I found out today:

-> When using two different installations sharing the same database: The problem still occurs (according to the proposal by trendoman » Thu Sep 12, 2019 12:33 pm), meaning the pages are blocking each other.
-> When using plain php files (no couch template) the problem does not occur, pages are not blocking each other.
-> I tried removing every session_start statements within couch folder, but no success.
-> When running on two installations using different databases, no problem occur (as expected )

Following code was used which is easy to test on your own installation:

Long running template:

Code: Select all

<?php require_once( 'couch/cms.php' ); ?>
<cms:template title='Test long running template' executable='1' clonable='0' hidden='0' >
</cms:template>

<cms:php>
   sleep(5);
   echo date('l jS \of F Y h:i:s A');
</cms:php>

<?php COUCH::invoke(); ?>

Quick running template:

Code: Select all

<?php require_once( 'couch/cms.php' ); ?>
<cms:template title='Test quick running template' executable='1' clonable='0' hidden='0' >
</cms:template>

<cms:php>
   echo date('l jS \of F Y h:i:s A');
</cms:php>

<?php COUCH::invoke(); ?>

So my conclusion is that CouchCMS is blocking simultaneous script execution because of database access.

Anybody any idea for a solution?

Regards,
Olliwalli

[KK]

Are you accessing the pages as super-admin?
Please try accessing as plain admin or as an anonymous visitor.
Let me know what you observe.

Thanks.

Hi KK

I tried accessing the pages as admin, but no difference.

Regards,
Olliwalli

[trendoman]

Oliwalli, many people were testing sleep in PHP and found that the whole website and other scripts were also waiting for the single sleep to end. The problem is in PHP's session.

test_1.php:

Code: Select all

<?php

//session_write_close();
sleep( $sec=10 );

test_2.php:

Code: Select all

<?php

echo "This should not wait!";

First, run the test_1 script in one browser's tab and then immediately run test_2 in another tab. Both will wait till the sleep completes. Next, uncomment line in test_1 and do the same - you'll see how test_2 immediately executes while test_1 continues sleeping.

P.S. I must withdraw my initial hypothesis that two installations will make a difference - it will not because the session is the same.

Trendoman, this makes no difference. The script blockes the other script when run in couch. When run in plain php it does not block. Also when making a <cms:pages ... call that lasts some seconds instead of sleep, the same problem.

Since this also accurs when running from different installation with same database, but not with different database I suggest that it has to do with database locking through couch.

[trendoman]

Trendoman, this makes no difference. The script blockes the other script when run in couch. When run in plain php it does not block. Also when making a <cms:pages ... call that lasts some seconds instead of sleep, the same problem.

Since this also accurs when running from different installation with same database, but not with different database I suggest that it has to do with database locking through couch.

Thanks for checking it out.
Another place where a lock may happen on a shared database is GC addon shipped with Mosaic. Try to disable it temporarily to see the effect by commenting out in /couch/addons/mosaic/mosaic.php::4 and report the result.

Code: Select all

//if( defined('K_ADMIN') ){ require_once( K_COUCH_DIR.'addons/mosaic/gc/gc.php' ); }