Dear Couchers,

Context: This is a branch off of the CouchCMS v2.3 topic, in which there was discussion on ways of using Couch alongside a static site builder such as Mobirise. Of course, Couch can already retrofit itself into any statically generated site. However, some designers may prefer building only with the frontend builder of choice (Mobirise, Blocs, or even Vue/React), and may not prefer retrofitting Couch, as this would be a one-way street; any subsequent visual updates they'd like to make via the site builder app would require overwriting and another round of Couch retrofitting

To solve this problem, perhaps, in addition to our traditional manner of retrofitting Couch, there is a way of using existing Couch concepts to let it serve as a headless CMS, not necessarily concerned with frontend view -- thus making itself compatible with anything capable of making HTTP requests, including Vue/React, anything that uses cURL, or any site builder that is capable of making/managing HTTP requests and responses.

- - - - - - -

Link to prior thread: viewtopic.php?f=5&t=13148&start=10#p37736

@KK, almost! But instead of Mobirise fetching the content and then exporting the resulting content statically, it would (1) fetch the content from Couch, (2) preview those results in the app, and then (3) export whatever frontend javascript is required to render what Mobirise was previewing in the app. Put another way, the content itself would not be rendered on a server (as with Couch) or even exported to html (as with Mobirise) but rather fetched and rendered in the browser itself by the javascript written by the code builder app (Mobirise in this case).

This exchange would be similar to how a frontend Vue or React application is meant to be independent of its CMS of choice (if it even chooses to pair with one). They reach out to any HTTP based API and then render that dynamic content in the browser. Similar to the relationship between HTML and CSS, this separates the concerns of (1) the frontend design (static html, css, k_is_page, etc) and (2) the content management (Couch admin panel, Contentful, headless WordPress, etc).

Many site builders (not just Mobirise) would benefit from an ability to write/manage GET requests to any headless CMS that is capable of delivering raw JSON. The site builder itself would simply need to allow its user to craft GET requests and subsequently be able to render those results in the app and export whatever javascript is needed to render those results in the browser.

This would allow any site builder designer to design in their app of choice, hook it up to Couch via API requests, export their project, and still be able to come back to their site builder app to re-design anything, export it again, and not worry about overwriting any Couch code since there was no retrofitting Couch in the first place.

In summary, I think static site builders of any sort (Mobirise, Vue/React, etc) would greatly benefit from a Couch backend that is capable of elegantly delivering its contents in the form of JSON. Couch developers already love building sites with Couch. Being able to share the contents of a Couch site in raw JSON opens many doors for Couch developers to work in tandem with designers who use site builders or develop with Vue/React. Of course, the site builders need to be capable of designing GET requests and rendering their responses, which is not Couch's responsibility. Maybe Mobirise will develop an extension for this, and maybe there are other site builders that have this broad capability. Vue and React are already designed for this concept.

- - - - - - -

In fact, I've begun dipping my toes in this already, letting go of k_is_page and k_is_home in many cases. I've been working with Vue lately to build frontend markup based on parsed responses it gets from GET requests to custom routes that deliver JSON via <cms:content_editable 'application/json' />.

Using this approach has been nice and is keeping my projects tidy. It's pretty straight forward with GET and POST requests that are meant for public consumption and input. The only tricky part so far is handling richtext output and certain CRUD operations where security would be of concern. I usually stick to CRUD via the Couch admin panel, but having access to this via HTTP requests (from anywhere outside the couch admin panel) would be door-opening.

I may post some examples of where I've found this technique useful, and if anyone else uses this technique or has thoughts, I'd love to see them. Especially concerning how a headless CouchCMS approach might work alongside mobile app developers who may want to interface with a Couch project in some fashion (may just public consumption/input, but maybe also authenticated requests via secret key/token in lieu of cookies).

All in all, I think understanding this concept would open a lot of doors for Couch developers to work alongside non-Couch developers, and there seems to be a lot of site builder and Vue/React folks out there who may benefit from what Couch has to offer in the content management and ease-of-backend-development department.

I hope this sparks some interesting discussions and solutions

[KK]

I may post some examples of where I've found this technique useful

I'd be very much interested in seeing those (as, and I am sure, would be others as well).
I request you to please go ahead and post as many examples as possible.

@KK, certainly!

A big caveat so far is richtext fields, which are difficult to manage with JSON, so for projects that need richtext (like blogs) I still use Couch like normal, but for more data + dashboard oriented sites (especially if I want to fetch a lot of dynamic content without page reloads) the API endpoint approach works very well. That being said, I've seen some JSON rich text editors that I may try to work in somehow, including Editor.js.

For the endpoints, I'm taking inspiration from Stripe and JSON Server. Both are elegant and follow REST principles.

Let's say we have two templates: galleries.php (clonable) and api.php (routable). Ideally, the api.php template should have appropriate route validation and constraints set, as described in the custom routes documentation.

- - - - - - -

Some simple GET API routes:

Code: Select all

// api.php

<cms:route name='galleries__list' path='galleries' />
// api.php?q=galleries
// api.php?q=galleries&_limit=5

<cms:route name='galleries__show' path='galleries/{:id}' />
// api.php?q=galleries/1

Code: Select all

// Note: I prefer to break off the view into smaller files
// and embed them based on k_matched_route, i.e.
// galleries__list.html, galleries__show.html, galleries__post.html


// galleries__list.html

<cms:content_type 'application/json' />

<!-- get the limit query parameter from the URL -->
<cms:set gpc_limit="<cms:gpc 'limit' method='get' />" />

<!-- return JSON response -->
<cms:abort>
  [
    <cms:pages masterpage='galleries.php' limit=gpc_limit>
      {
        "id" : "<cms:show k_page_id />",
        "title" : "<cms:show k_page_title />",
        "description" : "<cms:show desc />",
        "main_image" : "<cms:thumbnail main_image width='1200' image />",
        "images" : [
          <cms:reverse_related_pages masterpage='images.php' field='gallery'>
            {
              "id" : "<cms:show k_page_id />",
              "title" : "<cms:show k_page_title />",
              "caption" : "<cms:show caption />"
            }<cms:if k_paginated_bottom><cms:else />,</cms:if>
          </cms:reverse_related_pages>
      }<cms:if k_paginated_bottom><cms:else />,</cms:if>
    </cms:pages>
  ]
</cms:abort>

// Note: I like to wrap my JSON output in <cms:abort> tags as that will return *only* what's inside.
// This means I can make <!-- HTML comments --> above or below without worrying about comments
// being output in the response, which would break the JSON (no comments allow in JSON)

- - - - - - - - -


// On the frontend, we can fetch and console log the response from the endpoint
<html>
  // body of html document
  <script>
    fetch("https://mysite.com/api.php?q=galleries')
      .then((response) => response.json())
      .then((json) => console.log(json));
  </script>
</html>

// Output in console:
 [
  {
    "id" : "219",
    "title" : "San Fransisco Bridge",
    "description" : "A black and white gallery of the San Fransisco bridge",
    "main_image" : "https://mysite.com/couch/uploads/image/galleries/sfb/bw1-1200x900.jpg",
    "images" : [
      {
        "id" : "423",
        "title" : "Photo 1",
        "caption" : "Traffic enters the bridge"
      },
      {
        "id" : "424",
        "title" : "Photo 2",
        "caption" : "Fog of morning with birds"
      },
      // etc...
    ]
  },
  {
    "id" : "220",
    "title" : "Cafes of San Fransisco",
    "description" : "32 film photos of cafes in the San Fransisco area",
    "main_image" : "https://mysite.com/couch/uploads/image/galleries/sfc/photo-1-1200x900.jpg",
    "images" : [
      {
        "id" : "829",
        "title" : "Photo 1",
        "caption" : "Morning couple sits outside of an Italian cafe"
      },
      {
        "id" : "830",
        "title" : "Photo 2",
        "caption" : "A white dog sips water from a bowl outside of a cafe"
      },
      // etc...
    ]
  },
  // etc...
]


- - - - - - - - -

// galleries__show.html
// Very similar to gallery__list.html, but returns a
// single object instead of an array.

<cms:content_type 'application/json' />

<cms:abort>
  <cms:pages masterpage='galleries.php' id=rt_id>
    {
      "id" : "<cms:show k_page_id />",
      "title" : "<cms:show k_page_title />",
      "description" : "<cms:show desc />",
      "main_image" : "<cms:thumbnail main_image width='1200' image />",
      "images" : [
        <cms:reverse_related_pages masterpage='images.php' field='gallery'>
          {
            "id" : "<cms:show k_page_id />",
            "title" : "<cms:show k_page_title />",
            "caption" : "<cms:show caption />"
          }<cms:if k_paginated_bottom><cms:else />,</cms:if>
        </cms:reverse_related_pages>
    }<cms:if k_paginated_bottom><cms:else />,</cms:if>
  </cms:pages>
</cms:abort>

// Frontend fetch
<html>
  // body of html document
  <script>
    fetch("https://mysite.com/api.php?q=galleries/219')
      .then((response) => response.json())
      .then((json) => console.log(json));
  </script>
</html>

// Output in console
{
  "id" : "219",
  "title" : "San Fransisco Bridge",
  "description" : "A black and white gallery of the San Fransisco bridge",
  "main_image" : "https://mysite.com/couch/uploads/image/galleries/sfb/bw1-1200x900.jpg",
  "images" : [
    {
      "id" : "423",
      "title" : "Photo 1",
      "caption" : "Traffic enters the bridge"
    },
    {
      "id" : "424",
      "title" : "Photo 2",
      "caption" : "Fog of morning with birds"
    },
    // etc...
  ]
}

For simple text data (titles, captions, numbers, image URLs, etc), I find this to be an elegant way of interfacing with Couch from any frontend or even command line interface via curl.

Code: Select all

// in a unix shell, make a curl request and pipe the
// response to the jq tool (json formatter for command line)

curl https://mysite.com/api?q=galleries/219 | jq

{
  "id" : "219",
  "title" : "San Fransisco Bridge",
  "description" : "A black and white gallery of the San Fransisco bridge",
  "main_image" : "https://mysite.com/couch/uploads/image/galleries/sfb/bw1-1200x900.jpg",
  "images" : [
    {
      "id" : "423",
      "title" : "Photo 1",
      "caption" : "Traffic enters the bridge"
    },
    {
      "id" : "424",
      "title" : "Photo 2",
      "caption" : "Fog of morning with birds"
    },
    // etc...
  ]
}

Other examples of where an API endpoint approach comes in handy is for login and contact forms. Indeed, what's been shown here is just the tip of the iceberg.

Login form can check whether the email exists, password is correct, etc, and display a loading spinner while this request is being made asynchronously via the javascript fetch API and alert the user the relevant info without reloading the page. I may post the login form example here for review since it does entail some security considerations.

Contact forms can submit a POST request via javascript fetch API, and, similarly, alert the user with relevant info without reloading the page, i.e. if their information was sent successfully.

But again, the big benefit here is not that we can fetch data without reloading (we can already do that with jQuery AJAX). The benefit of Couch API endpoints is that *any* frontend application can interface with Couch, including Vue/React apps, as Couch is not necessarily responsible for rendering the HTML markup. Of course, you still can use Couch to render HTML, but having an API on the side will make the data available elsewhere and maybe tidy up the Couch project itself with re-usable functions that return endpoint data that you can call upon from other parts of the site.

[KK]

Thank you very much Very helpful indeed.
I wonder if you have any examples of POST?

[trendoman]

Interestingly, I do not see a justification for a separate routable api.php. Maybe you could expand on that? My standing is that Couch already handles routes perfectly -- Folder View, Archive View (with a ton of helpful Variables available in Views) -- you'd need to recreate views in api.php and still won't have the variables. so why the hassle?

Code: Select all

// This will be enough.
https://mysite.com/galleries.php?p=219&as_json=1

More than that, tag <cms:smart_embed> recognizes views and can automatically embed a tailored snippet with json generator.

Code: Select all

<cms:abort>
    <cms:content_type 'application/json' />
    <cms:smart_embed 'json-response' />
</cms:abort>

An API w/POST is a giant step towards better frontend/console editing features.

@trendoman, indeed! I do go back and forth on whether custom routes are the better route here vs default k_is_page, etc. You're right, the default views do come packed with handy variables (especially archive/folder view), but my intuition is that custom routes -- while more expensive to set up initially -- will always give more flexibility and control in the long run via route constraints and route validators.

They're certainly overkill for the simple gallery example I provided, but perhaps the ability to declare custom routes down the road will come in handy in ways I can't predict. My gut feeling.

One other benefit that I believe you only get with custom routes is the filter option (i.e. <cms:route name='list' path='premium-galleries' filters='premium-subscriber' />). Yes, I can check for k_logged_in or some other token passed along to the view to authenticate. However the premium-subscriber filter may apply to many routes beyond the premium-galleries. Furthermore, if I ever wanted to change the logic for what constitutes someone passing the premium-subscriber filter, I'd only need to update the logic in that filter file (not across the potentially numerous views that need the filter applied).

Maybe that's achievable outside of custom routes though? I'd love to see a thread doing a deep dive on pros and cons of using custom routes, route validation/constraints, smart_embed, URL query parameters, etc.

In the meantime, I do like the simplicity of "as_json=1" and the implicitness of <cms:smart_embed /> for embedding custom views. I'll have to implement that sometime and get a feel for that approach. Thank you!

And I'll post some POST examples soon!

POST example

Here's a music lesson inquiry example that accepts POST requests and dynamically responds to them with relevant, real-time JSON data based on the content of the request and the data we have in our database.

The endpoint:
First, we create a Couch endpoint like https://mysite.com/api/lessons/inquire. Personally, I'd use custom routes to define this and other related endpoints, in an api.php Couch template, but there are other ways of mapping URLs to our Couch code. As long as you can run Couch code when someone gets there; that's up to you.

Accepting input:
Now, given that endpoint, any client can post data to it. The client could be our own Couch site, someone else's WordPress site, a React frontend, an iOS app, or even a command line application via curl. We can respond to these requests by gathering the posted values (their question) and then running it through our system to respond with a raw JSON response (our answer). It's up to them to use that JSON however they see fit, and it'd be wise of us to offer good documentation to the developers of those sites how to interact with our little API (what POST data is expected of them, what types of responses they can expect from us, etc).

Business logic
In this example, I'm letting any client request music lessons via our Couch site. Some music lessons are likely to be handled rather quickly due to popularity or having plenty of teachers available (piano, voice, violin). Other requests may take longer to process due to teacher overload or other availability issues (oboe, harpsichord, Mongolian throat singing). We can dynamically let the client know what to expect based on the real-time teacher data available to our Couch backend.

Here's some Couch code for our endpoint. Do note: I include code comments here for Couch forum members, but JSON does *not* allow comments and will fail if any get leaked in. That being said, you can still comment your Couch code. Just be sure to wrap your intended output in <cms:abort> tags to ensure that only the encapsulated content is output (not your comments).

Code: Select all

// https://mysite.com/api/lessons/inquire
// Declare this as a JSON response
<cms:content_type 'applicaton/json' />

// Capture the posted values
<cms:set post_name="<cms:gpc 'name' method='post' />" />
<cms:set post_email="<cms:gpc 'email' method='post' />" />
<cms:set post_study="<cms:gpc 'study' method='post' />" />

// Filter the study by teacher availability. We'll assume the posted
// study came from a dropdown list of our available studies so we 
// won't expect anything else. Maybe the form dynamically GETs
// the available studies via some other GET endpoint of ours such
// as https://mysite.com/api/studies that returns a list of current
// studies we offer. We can also catch exceptions via <cms:no_results>
<cms:pages masterpage='studies.php' page_title=post_study>

  <cms:set study_id=k_page_id scope='global' />
  <cms:set teacher_count="<cms:reverse_related_pages masterpage='teachers.php' field='teaches' count='1' />" />

  <cms:if teacher_count lt '1'>

    <cms:abort>
      // Persist user to waitlist here (maybe using <cms:db_persist />)
      // Persist lesson inquiry record
      {
        "status" : true,
        "message" : "At the moment, we don't have any <cms:show study /> teachers, but we have placed you on the waitlist. We'll be in touch as soon as another <cms:show study /> teacher is available."
      }
    </cms:abort>

  <cms:else_if teacher_count lt '3' />

    <cms:abort>
      // Send student info to teachers or another intermediary here (maybe using <cms:send_mail>)
      // Persist lesson inquiry record
      {
        "status" : true,
        "message" : "We're passing your information along to a <cms:show study /> teacher. Please know, we have limited teachers in this area. So it may take some time before we can pair you with a teacher. You can expect some response from us within a week."
      }
    </cms:abort>

  <cms:else />

    <cms:abort>
      // Send student info to teachers or another intermediary here (maybe using <cms:send_mail>)
      // Persist lesson inquiry record
      {
        "status" : true,
        "message" : "We're passing your information along to a <cms:show study /> teacher. We have plenty of <cms:show study /> teachers, so you should hear back very soon!"
      }
    </cms:abort>

  <cms:if>

  <cms:no_results>
    <cms:abort>
      // Persist lesson inquiry record
      {
        "status" : false,
        "message" : "Sorry, we don't offer <cms:show post_study /> lessons at the moment."
      }
    </cms:abort>
  </cms:no_results>

</cms:pages>

- - - - - - -

Couchmembers, I'd love to hear your own perspectives or comments

[trendoman]

And I'll post some POST examples soon!

..POST example

Pardon me, but I fail to understand what this topic is really about. We've had a few examples already on howto get some json response based on <cms:pages>, whether request is formed as GET or POST -- preferring one over another does not make any difference from backend POV.

I hope there's more!!

P.S. Wrap comments in <cms:ignore> and save time on warnings

@trendoman, yes I've noticed you use <cms:ignore> like this, and I have taken on that approach where I really need it But only where I really need it. I don't like wrapping every comment area in its own tag. Personal preference.

- - -

The lesson inquiry example is very simple (intentionally so). Perhaps a more clear use case would be POSTing to authentication-protected URLs, to take authenticated action such as creating a blog post, which is not something meant for general public interfacing (as with the simple lesson inquiry example). Consider the use case of Couch as a server application capable of granting privileged access to particular clients to take authenticated action via API keys. These clients could be WordPress sites, iOS apps, command line tools, etc and the content they'd be interfacing with could be blog posts, user settings, etc.

For instance, something I've been considering lately is providing my web business clients a place to manage their account with me (updating their billing info, managing contracts, etc.). But instead of providing them with "yet another login" to my site, I provide their site "access" to interface with my site (both GET and POST). So they log in to their existing site, using their existing login, and have an area to administer their data that I keep central on my site. Where we'd normally use something like <cms:if k_logged_in>, we'd instead use <cms:if client_provided_a_valid_api_key>. We'd need to have those API keys generated on the central site and dispersed to the particular clients securely, much like how Stripe provides API keys that can be expired or created as needed.

Essentially, this builds a bridge between two Couch sites (or any other site) to interact in as many ways as normal users interact with normal sites (it's really the same thing: a client and a server). You could even "lock down" areas of a remote site to handle subscription based features from a central site. That would take a lot of careful planning and thinking through edge-cases, but it's theoretically possible. Not something I'm focused on today, but maybe in the near future.

Basically, the Couch developer experience + a well-designed API = creative potentials we can't easily foresee.

[trendoman]

I agree, personal comfort is king. I once devised an addon stripping comments in pre-process, however found that was a working but silly idea. There is a nice thing about Couch tags - they ignore unknown params. This can be used for comments! I notice you commented below <cms:abort> - here is a tip to NOT use cms:ignore and still be able to keep comments in source code undisturbed -

Code: Select all

<cms:abort comment='
    // This section is about to exploode!
'>

Simple things yet joyful.

- - -

I'll allow myself just briefly express some ideas regardless of the topic, but relative to Couch.
If I had any clients, I'd think more in the way to let them interact with CMS in modern ways -

• use chat bots (Telegram, Whatsapp etc) to edit their data and request updates
• authenticate to website via a secure phrase or question-reply in a chat window (no password)
• send content to CMS via email to have it auto-published instead of forms

Offtopic: I also believe the privacy issue will be dealt with in future by trying something like Youtube-in-reverse, where private content is shared with youtube from user's store and is permitted to be distributed upon conditions (a content owner can exclude certain search queries or countries, for instance). Maybe it is even served from user space and only temporarily cached at youtube's. Couch could be an easy way to create that 'internet persona' and easily manage content that is permitted to be reposted/published on social media later (networks will request content via API from user).

P.S. I very much doubt Couch will handle an iOS app with many (over 100) simultaneous users. It is slow and drops database connection in my tests unexpectedly after a light load on an empty template. CouchCMS needed a serious refresh long ago (2 years since my tests with Apache Bench)!