Hi,
I've been stuck on this issue for a while. Maybe it's something very simple that I'm overlooking.

I work with a large set of datapoints (written separately in pages), I try to write all of them into a large json file. I write them one by one. Before I write them to the file I do a DB persist, to set an editable "added_to_json" to "1". I loop through the pages where "added_to_json" is not "1". But I keep getting double entries.

Code: Select all

<cms:pages masterpage=k_template_name custom_field="added_to_json!=1" limit="1" >
   <cms:db_persist _masterpage=k_template_name _mode="edit" _page_id=k_page_id added_to_json="1" >
      <cms:if k_success>
         <cms:write "file.json" >...</cms:write>
         
      </cms:if>
      <cms:if k_error>
         <cms:show k_error/>
      </cms:if>
   </cms:db_persist>
</cms:pages>

[KK]

Hi,

By "double entries", do you mean that, when you run the script again, the page with 'added_to_json' set to 1 in the previous run shows up once more?

Please clarify.

yes exactly, I run the page multiple times in order to write all the pages into the file and sometimes I get double entries.

[KK]

TBH, I am unable to see what could be causing this.. perhaps you are using the 'staggered' approach of processing the pages in batches and one of the runs abends (this will cause all saves in the run to be rolled back)?

Cannot say anything for sure without seeing your code in action, I am afraid.

[trendoman]

Using added_to_json!=1 means that a non-strict search is performed, which may (or may not) cause unexpected issues. With fixed values it is more reliable and also faster to use !== op. Mind that in your case the code would better be "added_to_json==" without any value.