[trendoman]

Hello, KK!

In most cases of high-profile websites, the content is updated dynamically. This includes multilanguage sites as well. However, parts of the content rarely changes. Ex: navigation, footers with navigation, forms, relations etc.

It would be nice to improve on these calcs, while the cache is disabled in site config, using something like:

Code: Select all

<cms:cache >

...one-timer heavy runner...

</cms:cache>

It would create explicitly cached chunks of html output.

Let me know if this is a valuable suggestion.

[KK]

Hi trendoman,

Yes, indeed, it is a valuable suggestion.
However, I must add, that this feature hides many a nuance within it and won't be a straightforward thing to implement.

That said, at some point we'll have to get something like this in Couch.

[trendoman]

Happy

Till then, it is already possible to perform the simplest trick - copy-paste the generated html from the frontend to a snippet (to let templates stay clean).

[trendoman]

A follow up from here viewtopic.php?f=2&t=10213&p=24528

I wish we had some smart tag for that:

Code: Select all

<cms:set_cache 'city >
 
       ..cms:pages or any queries..

</cms:set_cache />

And call:

Code: Select all

<cms:get_cache 'city' />

A follow up from here viewtopic.php?f=2&t=10213&p=24528

I wish we had some smart tag for that:

Code: Select all

<cms:set_cache 'city >
 
       ..cms:pages or any queries..

</cms:set_cache />

And call:

Code: Select all

<cms:get_cache 'city' />

The problem with this RE: Caching the snippet list so that it isn't re-generated each time, that at least I can see..

AFAIK the cache just generates a html version of each cloned page/template final output (a static html file), but it doesn't create separate files for each of these (So your 100kb snippet) because this just doesn't make sense for the generic caching couch does.

I can also see a useful feature in what you want to achieve, though. Caching just that snippets output would allow much faster implementation later. The issue is snippets can have almost anything in them which makes it difficult to setup in a way that couch knows when to invalidate the cache (Other than the 24 hour period). Currently it does this whenever a super-admin saves the template/page in the admin panel. How would couch know when a snippet-cache needs to invalidate? This is the biggest question I could see causing issues with any kind of implementation.

Still, it would make for a nice addon to advanced users, in my opinion.

The other issue I see is actual use-cases for this are small, because in the end you should be aiming to use the cache for every page (And then caching individual snippets is useless) as the entire template/page would be served from its cached version. Wordpress caching plugins almost all work in the same way (I'm sure there are more advanced versions that allow this kind of functionality). It seems more useful for back-end admin testing, I could see cases where setting this up to make testing changes to templates with large embedded snippets far more time-saving than just loading the template outside of the cache.

I can't see many times where you would need NOT to cache the entire template and just snippets (That said, I can see a few). This would be a neat addon for couch imo.

[trendoman]

I want this to work alongside proven Couch Cache mechanism.

The whole point of normal snippets is that people usually write->create->place them manually. Couch embeds snippets and caches them as usual.

Now, create a new template, register it and place some very exhausting code inside the pair of new tags. Visit it once as super-admin and Couch will create the snippets in /snippets/cache directory. These snippets will contain the html-output of the code.

Place the calling self-closing tag anywhere else in website and Couch will treat it just as any embedded snippet, effectively placing it in the resulting html, invalidating as per default routines.

Template with hard-nut codes can be revisited upon your decision, as super-admin, if you sense that the appropriate time has come (new data appeared, changes had been made etc). Does it differ from making changes to a manual snippet? No, because your generated snippets are just the same as normal ones. Couch only made it a way better in management.


P.S. As to use-cases - there are some. Registered and logged users normally have cache disabled by Couch. Registered users may have huge forms to fill with multitude dropdowns. Another bleeding sample is a 'cart' floating on every page in shop, as it effectively requires <cms:no_cache />, etc, etc. Also, any regular cache invalidation can bump the first visitor and hit hard the server, with 5 sec per one heavy request Vs 0.2 sec for embedding the same piece. I can only hope our prays are heard

[trendoman]

This topic came to life again recently, while I am working on a very demanding project. Of course, a high-load website with multiple calculations needs proper caching, apart from somewhat light-weight Couch caching system.

So far, with absence of solution with snippets (as above) there had to be found another way. Let's say we need to show on map points of interests, related to the place. Let's also put some constraints on it - a map has a legend which could look like this:
[restaurants] - 300 restaurants in this area
-- [vegetarian r] - 30
-- [mexican r] - 50
-- [chinese food r] - 150
-- [international cousine r] - 50
-- [other r] - 20

The same goes for
[shopping malls]
[recreational amenities]
[other local businesses]
[public services]

The idea is that each main category shows localities, filtered by both - location and some subcategory. And the problem is that amount of calculations is massive so that a page can be generated in 60 seconds with default tags or 10-15 seconds with optimized settings for <cms:pages> and 5-7 seconds with direct <cms:query> which bypasses default routes and gets data from database directly with sql request.

It only can get worse from this, when map is zoomed out, when user chooses additional filters [currently opened; average paycheck; good reviews; public transportation available].

So, for such a situation one could see a multi-server approach with tens of thousands of investments into hardware and programming. CMS is expected to be some very enterprise-like to handle such a task The fact is this also can be done still keeping ties with couch and having smart caching mechanisms. One of solutions could be cached snippets of pregenerated datasets. It has some cons. Another solution I discovered recently and is implementing. Caching is performed in form of pregenerated tables in mysql database. Users still send requests the normal way from the front-end with Couch <cms:query>, but data is served from caching tables and not regular monstrous couch_pages table (yes, it holds all cloned pages from the whole website).

Dynamic requests to tables are performed 10-100x faster than common fetching. The salt of this approach is ability to place triggers in mysql, so that when any info in admin panel is updated/deleted/inserted - then relevant info is updated/deleted/inserted in those generated tables. This makes it very plug-and-play approach - set up everything once and it works. This is called materialized view in other databases. Google for it, because it's really cool feature. Mysql doesn't support it though, but there are ways to bend it and make behave like more established rivals.

Hope this helps someone who suffers from very slow operations on server.
I'd gladly prepare some documentation and tutorial for this once I'm through and finally can have some time.

P.S. Currently database looks like this. I am working on extending the above approach and probably will ease also sorting, as sorting by page names is incredibly demanding for a million rows.

Image 030.png (8.59 KiB) Viewed 8583 times

Another useful feature would be the opposite of the <cms:cache> tag...a <cms:dont_cache> tag...which would do exactly what it sounds like. Basically given a page that will be cached, always generate anything between the <cms:dont_cache> and </cms:dont_cache> tags even when the rest of the page is cached.

[trendoman]

Temporarily resolved by saving piece of code in session as in http://www.couchcms.com/forum/viewtopic.php?f=5&t=7377

Code and calcs still run once when user visits page in browser, but on page reload session is polled and no hard calculations needed. Session saved me, it's much faster and limited only to disk i/o (reading saved session file).