Coveo for Sitecore - Endpoint deprecation issues and resolution - TLDR

Coveo planned to deprecate their Coveo for Sitecore endpoint and requested clients to use Organization specific endpoint for better control over traffic and rate limiting. I wrote a blog article earlier and provided the steps to migrate to the organization specific endpoint. 

Coveo for Sitecore - Endpoint Deprecation & Migrate to Organization Specific Endpoint: https://www.nehemiahj.com/2024/08/coveo-for-sitecore-endpoint-deprecation.html

Even after migrating to the right organization specific endpoint, we were still informed that we are using the deprecated endpoint and we may have issues to analytics and search related API calls. In Coveo Administration Console, Visit browser / event browser feature does not show the endpoint being used for the incoming traffic. 

So every time, we wanted to test whether it reaches the right endpoint, we worked with Coveo support and they provided the results. Results were contradictory. Sometimes it reached the right endpoint and sometimes it reached the wrong one 🤷‍♀️. Our setup is described below.

• We have Cloudflare as our entry point. 
• Azure Front Door is used for routing as we have multiple legacy sites and Sitecore sites. 
• Sitecore site is hosted in Azure VM as IaaS setup. 
• Coveo for Sitecore module is used to setup the Coveo related pages. 

As Coveo support requested, I did few changes to Cloudflare and Azure Front Door.

• Bypass Cache for Coveo requests in Cloudflare. Bypassed for any URL path /coveo/rest/*
• Disable cache and WAF in Azure Front Door - We didn't use WAF and Cache in Azure Front Door. So no changes were implemented. 

Scenario 1: Disabled the Sitecore proxy

Disabled the Sitecore proxy through the command center. How to do it: https://docs.coveo.com/en/3401/coveo-for-sitecore-v5/disable-the-reverse-proxy.

Result: All Coveo requests reached the correct organization endpoint. I could even see it in the network tab as Coveo requests are routed directly from client browser to Coveo organization endpoint domain. 

Scenario 2: Enabled the Sitecore proxy & requests from internet

This is a regular scenario where proxy is enabled in Sitecore and requests were made from internet. 

Result: All Coveo requests reached the deprecated endpoint. Coveo requested us to implement Bypass cache in Cloudflare and disable cache & WAF in Azure Front Door as mentioned above. But Coveo support mentioned that requests are sent to deprecated endpoint. 

Scenario 3: Enabled the Sitecore proxy & requests sent inside Sitecore Server

This was just a random request I made and Coveo support checked their side whether it is sent to deprecated endpoint or not. 

Result: All Coveo requests reached the correct organization endpoint. 

Coveo support brought in core engineering team to the discussion and they were skeptical that some rules in Cloudflare or Azure Front Door transforms the domain to the wrong domain. I was sure that there is no outbound rules which could have caused this issue. They asked us involve our network team to investigate as there is no problem with Coveo. 

In-Depth Analysis

Enabled Coveo Debug Logging:

We enabled Coveo Debug logging to understand the domain which is used to send the Coveo proxy request. To enable debug logging, you can modify the <log4net> <logger> section in this file \App_Config\Include\Coveo\Coveo.SearchProvider.Custom.config. 

After enabling debug logging, Coveo related logs were captured in Coveo.Search.txt file. It clearly showed that we are sending the traffic to Organization specific endpoint (which is the correct endpoint) in all the scenarios. We had to prove that to Coveo by other means. 

Captured Coveo Header Information with Domain:

Since I cannot install any sort of network tools in the server, I created a simple .NET app to read the incoming request and log the header details in a text file. I hosted it in a separate development Azure VM with public access. The bindings for this app are Coveo deprecated endpoint and Coveo organization specific endpoint domain. In the Sitecore Azure VM, I added the Coveo organization endpoint domain and deprecated endpoint domain in the server host file pointing to the development VM. 

So any requests to Coveo deprecated endpoint and organization endpoint will reach the app hosted in the development VM. I logged the entire request header information in both the scenarios. From my side, I confirmed that all the requests are having the right endpoint so it is not our problem but something to do with Coveo network side. We asked Coveo support to involve their network team and we shared the entire log information. 

End Result

Coveo Support found the issue at their side. When the Sitecore proxied request reached Coveo and if it has X-Forwarded-Host header, then Coveo considered as invalid request and routed all traffic to deprecated endpoint. The request did not have X-Forwarded-Host header if we send the request inside the server (without Cloudflare or Azure Front Door) so those requests were marked as valid and routed to organization endpoint. Unfortunately we do not have any control to customize the request to Coveo using Sitecore proxy as it is part of the Coveo for Sitecore module. 

On Nov13th 2024, Coveo support released the fix at their side and we gave a couple of Visit IDs to test and they confirmed that it reached the organization endpoint. Also, Coveo extended their deprecation deadline to one last time to June 3rd 2025. 

The X-Forwarded-Host (XFH) header is a de-facto standard header for identifying the original host requested by the client in the Host HTTP request header. Host names and ports of reverse proxies (load balancers, CDNs) may differ from the origin server handling the request, in that case the X-Forwarded-Host header is useful to determine which Host was originally used. - Mozilla Developer Docs.

Since we have Cloudflare and Azure Front Door, the X-Forwarded-Host header is added to the request and it reached Sitecore server. Coveo for Sitecore module uses the headers from the original request, transforms only a set of headers with different value and copies the rest of the headers from original request to the proxied Coveo request. 

Once Coveo made changes at their side, this started to work for every Coveo clients. 

Our traffic data is also moving towards organization specific endpoint.

Happy Customers 😊😊😊

XM Apps API - Use Case for large organizations

In one of my previous blog article, I discussed about the newly introduced XM Apps API to manage XM Cloud Apps. What could be the use case for these APIs? 

I worked for a multinational confectionery, food, and snack food company client that had over 160 brands. They were using Sitecore XP, .NET-based applications, and other technologies for their brand sites. One of the pain points for the brand managers was the inability to quickly set up, build, and launch promotional sites. About 10 years ago, we proposed a solution that allowed brand managers to request new Sitecore, .NET, or other tech stack sites through a centralized portal. With the approval of IT and other stakeholders, the setup for a new Sitecore site would be ready for the brand manager and their development agency in just 20 minutes. Previously, this process had an SLA of 3-4 days due to various manual tasks. We automated everything, reducing the setup time to 20 minutes, enabling the development agency to start developing using existing reusable components and launch the site within an hour.

With XM Cloud Site management features along with the XM Apps API, large organization can make use of centralized internal portal to tie up with XM Apps API to setup, launch and destroy sites in a matter of seconds to few minutes. With automation, there is a huge impact to the speed with low/no manual errors. 

Explanation

1. Brand manager wants to launch a promotional site for an upcoming event. 
2. Brand manager requests a new site using Centralized Site Request portal. As part of the request, budget, project code to charge to manage XM Cloud and other relevant platforms to host a site, development agency details with all the developer information including email, name and other details, project sponsor information are added. 
3. IT and other stakeholders of the organization review the request and approve it. 
4. The centralized site request tool is based on Task-Oriented Architecture. Adds the list of tasks in the queue. Agent (start with a simple PowerShell scripts) takes each pending tasks and executes it in the order as configured. 
5. With XM Apps API and the availability of other APIs, these tasks can be executed in the matter of seconds to minutes. 
6. Development agency will be informed automatically and development can be started immediately. 

Can It Be Done?

Certainly. Even with Sitecore XP hosted in IaaS with no support of APIs, we were able to bring down the Site setup SLA from 3 days to 20 minutes with this kind of automation 10 years ago. With the availability of XM Cloud APIs and other APIs, large organization can be benefitted to automate and manage huge number of requests with no manual errors. 

Happy Customers!

XM to XM Cloud Migration tool - 2.1 - Installation and Dependencies

In the previous article, I provided a brief introduction of XM to XM Cloud Migration Tool. In this article, installation, dependencies and installation steps are explained.   

Installation

• Once downloaded, extract the content in a folder. 
• In the source CM instance, install the package, SitecorePackage.zip, which is available in the extracted folder. 

• Once the package is installed, you will see a folder (_MigrationHarness) in the CM root folder and it has all the dependencies needed for the tool to work.
• Additionally, two configuration changes are necessary. 
• web.config: Add the following configuration under existing parent XML elements. 
  
  

• Sitecore.Services.Client.config to allow access to entity and item services. Instead of modifying the Sitecore configurations directly, you can apply the patch as below. Important: Remove this setting once the migration is over as it allows access to all entities and Item Services. 
  
  

• Run the executables in Admin mode and start migrating content. There are two executables. 
• XMMigration.exe - For migrating using CLI
• XMMigrationGUI.exe - For migrating using GUI

Uninstallation

Since the dependencies are contained in a separate single folder - (_MigrationHarness), uninstallation is simple. 

• Delete _MigrationHarness folder 
• Revert web.config change. 
• Remove the Sitecore patch file for Sitecore.Services.SecurityPolicy setting. 

XM to XM Cloud Migration tool - 2.1 - Introduction

XM Cloud: Sitecore’s cloud-native, SaaS enterprise-ready CMS that creates content once and delivers across any channel for unforgettable customer experiences. So many years, Sitecore CMS has been deployed in On-Prem and PaaS. Migration from one On-Prem to another On-Prem was harder but doable using direct database access where we can backup and restore it in the target environment and migrate the content. With XM Cloud as a SaaS product, this is not possible anymore. We need certain tools which can access XM Cloud SaaS platform and migrate the content. One of the tool created by Sitecore is XM to XM Cloud Migration Tool. It was first introduced in 2023 with the aim of migrating a large chunk of content (as-is) from On-Prem to XM Cloud. 

XM Cloud Migration Tool Image (AI-Generated)

XM to XM Cloud Migration Tool - v1 (Deprecated)

XM to XM Cloud Migration Tool version 1 was introduced in 2023. 

• Supports Sitecore v10.1+
• On-Prem only
• Uses Sitecore Content Serialization (SCS) and Sitecore CLI under the hood

This tool worked but it was slow since content needs to be serialized in the source system, migrated and then pushed into the target system. Many Sitecore customers who are using either in 8x or 9x are not able to use this tool for their XM Cloud migration. 

XM to XM Cloud Migration Tool - v2.1

The newly introduced v2.1 during the Sitecore Symposium 2024 has significant improvements and features. 

• Supports Sitecore v9.1+
• On-Prem and PaaS 
• Uses Google's Protocol Buffer format (Protobuf)
• No longer dependent on Sitecore CLI or Sitecore SCS.
• Dependencies are installed using a Sitecore package. 
• Dependencies are contained in a single folder (easier to uninstall)

Essential Facts to Be Aware Of

• Before migrating content, templates need to be migrated first. 
• Take caution in migrating System folders. Migrate only custom created items. 
• Supports only migrating media items stored in database or Azure blobs.

Though this tool was created by Sitecore, it is not well-documented in the Sitecore documentation portal (don't know the reason 😊) but markdown are available in the tools download zip. You can download the tool here. 

In the next article, we will discus on installation and uninstallation. 

Part 2: Coveo for Sitecore - ML Model - Product Recommendation (PR) - Prerequisites

This is the second part of the series where I write about Coveo ML model for commerce which offers personalized recommendations to enhance customer engagement. In the first article, we discussed various Coveo ML models, including the Product Recommendation (PR) model and its strategies.

In this blog, prerequisites for PR model is listed below.

Coveo Machine Learning Product Recommendation (PR) models leverage usage analytics (UA) events to get trained with user interactions. This will then recommend products based on the strategies used. Since these are related to products, commerce related UA events are necessary. 

Based on the strategies, required events are listed below.

• Strategy: Cart Recommender (cart)

This strategy is used in the cart page where recommendations are made based on the products which are already in the cart. This is based on the UA events from other similar users who either viewed the details page of the product and purchased the product along with the currently added cart products. 

UA Events: detail (viewed details page), purchase (completed the purchase of products)

• Strategy: Frequently Bought Together (frequentBought)

This strategy is used in the product details page or even in quick view page where recommendations are made to purchase along with the currently viewing product. 

UA Events: purchase (completed the purchase of products). Optional: add/remove product to/from cart. 

• Strategy: Frequently Viewed Together (frequentViewed)

This strategy is used in the product details page to recommend other products which are viewed along with the same product by other users.  

UA Events: detail (viewed details page). Optional: click (clicking product links from PDF or even from other recommended product lists)

• Strategy: Popular Items Bought (popularBought)

This strategy is used to recommend popular products based on the number of purchases happening at a particular period of time. This can be used in any section of the site where we want to showcase the popular items which are bought by other customers. Usually it is marketed as trending products. 

UA Events: purchase (completed the purchase of products)

• Strategy: Popular Items Viewed (popularViewed)

This strategy is used to recommend popular products based on the number of times a product is viewed at a particular period of time. This can be used in any section of the site where we want to showcase the popular items which are viewed by other customers. Usually it is used during a big sale event to showcase the popular items which are being viewed by other customers.

UA Events:  detail (viewed details page). Optional: click (clicking product links from PDF or even from other recommended product lists)

• Strategy: User Recommender (user)

This strategy is typically based on the behavior of the user. Recommendations are calculated based on user browsing pattern, product details page views, add/remove or purchase events. This can be used in any section of the site as it is purely based on the user's behavior. 

UA Events: detail (viewed details page). Optional: purchase (completed the purchase of products), add/remove to/from cart.

Based on the requirements, appropriate strategy can be used to get the recommended products. Even though we have multiple strategies, we can create one PR model to train the model with all UA events and then use separate searchHub to target a specific strategy and get recommendation based on the strategy. 

We will implement this in the next blog. 

Sitecore Symposium 2024 - My notes

Sitecore Symposium 2024 was a fantastic event this year! I had the chance to meet with Sitecore experts and learn about their latest products and plans. It was a great opportunity to network and stay up-to-date on the latest trends in digital experiences. I want to acknowledge CloudIQ Technologies for their support in making my participation in the Symposium possible. It was a memorable event.

As a Sitecore MVP, we got lot of insights about Sitecore's product decisions and quick preview of what is going to be shared in the Symposium in the MVP Summit. I was particularly impressed by the insights shared by the Sitecore leadership team. They talked about the future of digital experiences and how Sitecore is evolving to meet the needs of businesses.

Overall, Sitecore Symposium 2024 was a great experience. 

Highlights:

1. Sitecore Stream: A Game-Changer for Digital Experiences: Sitecore announced Sitecore Stream (in the keynote), a powerful new add-on module that can be integrated into any Sitecore ecosystem. This innovative tool is designed to enhance efficiency and productivity for all Sitecore users.

   Whether you're using Sitecore XP on-premises or in the cloud, Sitecore Stream can help you make better decisions faster. Its brand-aware AI ingests and analyzes all of your brand information, providing you with actionable insights that can be used to optimize your marketing campaigns.

   With Sitecore Stream, you'll have the power of AI at your fingertips, enabling you to make more informed decisions and achieve better results.
   
   

   
   
   
   

2. Sitecore DXP: A Strong Foundation for the Future: Despite the focus on cloud-based products and AI at Sitecore Symposium, it's clear that Sitecore DXP is still a vital part of the company's strategy. Many customers expressed interest in the future of DXP, and Sitecore is responding by investing in its development.

   Sitecore Stream is an add-on for XP as well. Sitecore plans to introduce new add-ons for DXP, along with cloud-based tools, to help customers modernize their XM and XP platforms. This approach will allow Sitecore to continue innovating while providing flexibility for customers who prefer on-premises solutions.

   The upcoming release of Sitecore 10.5 in October 2025 is another positive sign for DXP. Sitecore is also improving its XM to XM Cloud migration tools to make the transition easier for customers.
   
   

   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   

3. Sitecore XM Cloud: Rapid Improvements: 
   

   
   

   
   
   
   
   HIPAA Compliance:
   • This is a crucial factor for FinTech companies, ensuring they can safely handle sensitive customer data.

   Modern Tools:

   • Pages, Site Management, and Forms provide a robust foundation for building and managing digital experiences.
     
     
     

   Performance Enhancements:

   • The new publishing mechanism and support for WebP format contribute to faster load times and improved user experience.
     
     
     
     
     
     

   Data-Driven Marketing:

   • Analytics insights within Pages empower marketers to make informed decisions and optimize campaigns.
     
     
     
     
     
     
     
     
     

   Generative Copilots:

   • These AI-powered tools streamline campaign setup, A/B testing, and publishing, making it easier for marketers to create effective content.
     

   Open-Source .NET Rendering SDK:

• This is a significant advantage for existing .NET customers, allowing them to leverage their existing skills and resources for XM Cloud migration without requiring extensive training in Next.js.
  
  
  
  
  
  
4. Developer Experience - Freedom of Choice: It has been a central focus at the Symposium. Sitecore's framework-agnostic approach, unlike solutions tied to a specific framework like Next.js, offers developers the freedom to choose the tools and technologies that best align with their project requirements.
   
   
5. OrderCloud: Trade-in, refurbished products and second-hand products have become increasingly popular in recent years. Though OrderCloud was not a prime showcased product in Symposium, Sitecore is working on feature like this. 

Sitecore XM Cloud CLI Cheatsheet updated

I created couple of cheat sheets for XM Cloud CLI earlier this year and it has been viewed more than 5000 times and it has been downloaded more than 1000 times. To keep the cheat sheet up to date, I have updated the cheat sheet and published it. 

To access XM Cloud Cheat Sheet, refer these two links below.

Sitecore XM Cloud CLI Commands List Cheat Sheet by nehem87

Sitecore XM Cloud CLI Cheat Sheet by nehem87

Introducing Sitecore XM (Cloud) Apps REST API

While XM Cloud Deploy APIs are useful to manage Deploy Apps related activities, Sitecore XM Cloud team has released a new set of APIs to manage XM Cloud Apps. This newly introduced APIs can help us to manage the XM Apps collections, sites, analytics, and languages. The online API console released by Sitecore works more like a postman (like in OrderCloud API console) and enables us to use the browser to send the request the environment. 

In order to use this API, JWT token needs to be created. In order to do that, we need to create the automation client in Deploy App and then /oauth/token endpoint is used to get the JWT token along with client id and client secret. Please note that to create client credentials in Deploy App, we need to have Organization Admin or Organization Owner access to Cloud Portal. 

XM Apps API:

• Sites
• List sites
• Create a site
• Retrieve a site
• Update a site
• Delete a site
• Duplicate a site
• Rename a site
• List sites that use an analytics identifier
• Unassign analytics identifiers from sites
• Retrieve the hierarchy of a site's main page
• Retrieve the hierarchy of a page
• List ancestors for a page
• List children for a page
• List hosts for a site
• Create a site host
• Retrieve a site host by host ID
• Update a host by host ID
• Delete a site host by host ID
• List site templates
• Upload a thumbnail for a site

• Collections
• List site collections
• Create a site collection
• Retrieve a site collection
• Update a site collection
• Delete a site collection
• Rename a site collection
• List sites in a site collection

• Jobs
• List site job statuses
• Retrieve a job status

• Languages
• List languages
• Add a language
• List supported languages
• Update a language
• Delete a language

API Console Features:

• Try it functionality to test the API via browser itself. 
• Provides a request payload with explanation of each attributes.
• Ability to set the environment variables
• Ability to set the authorization token.

Try it here. 

Coveo for Sitecore - Using Advanced Field Queries

In Coveo for Sitecore, there are scenarios where we need to create advanced complex queries to get specific results. Though we can achieve most of the search queries using basic queries, advanced queries helps us to narrow the search results. 

When dealing with Coveo Advanced field queries, 

• queries are performed against field values directly and not through the index. 
• can search for special characters in field values. In basic query (which is through index), special characters are not searchable. 
• can be only searched 'with special characters' against fields that have Facet options or Free text search options enabled. 
• drawback with advanced query is that response increases if the number of fields increases, field values and complexity of the query expression. 

Advanced Field Query Syntax

Advanced Query Field Operators

Wildcard Match (*=)

• Handles any wildcard characters (?, *).
• Can use one or more wildcard characters. 

Example: Intel processor containing I9

@processor *= "*I9"

Fuzzy Match (~=)

• Expands search results by permitting slight difference between search query and results. 
• Matches 20% difference from the search query.

Example: "Gigestive Capsules" results in "Digestive Capsules"

@medicine ~= "Gigestive Capsules"

Phonetic Match (%=)

• Transforms query into Phonetic codes, then results that shares similar phonetic code. 

Example: Flour results in Flour and Flower

@type %= "Flour"

Regular Expression Match

• Matches with the regular expression to get accurate results.

Example: Field containing URL with a subdomain URL. - https://prod1-cm.examples.com/product

@clickableuri /= "^https:\/\/[a-zA-Z0-9.-]*\.examples\.com\/.*$"

Scenarios

Sitecore Commerce - Refresh Commerce Cache - This operation is not available unless admin mode is enabled: FLUSHDB

In one of the eCommerce application, Sitecore Commerce 10.1.1 is being used. After few infrastructure consolidation, testing out Refresh Commerce Cache option in CM resulted in the below error. 

Based on the exception, we found that we can add AllowAdmin option to the Sitecore Commerce Redis Connection string. All Commerce related settings are available in this folder - \App_Config\Include\Y.Commerce.Engine.

In Sitecore.Commerce.Engine.Connect.config, allowAdmin=true configuration is added to the Redis Connection string. Refreshing the commerce cache worked without an error. 

Sitecore Scheduled Publishing Module - Published 10.4 packages and Docker images

Sitecore Scheduled Publishing module has been updated to support Sitecore 10.4. Following changes were made as part of the release. 

1. Updated Source to Sitecore 10.4
   
   Module source has been updated to 10.4. 
   
   
2. Created Packages for Sitecore 10.4
   
   Two packages were created. One is with IAR files for content and another one is the regular Sitecore content package. 
   
   
3. Created and published Docker Images
   
   Created and published new Docker image with tag 10.4. This image is built with IAR files. Along with 10.4 tag, latest tag has been updated to 10.4. You can view it here in Docker Hub. 
   
   
4. Documentation updated for schedular configuration
   
   In order for the scheduled publishing to publish the content, Sitecore's scheduling frequency and interval needs to be adjusted. This has been clearly documented in Readme markdown.

A new release has been added in GitHub with all the relevant assets. https://github.com/nehemiahj/SCScheduledPublishing/releases/tag/Sitecore_10.4

Troubleshooting Issues in Sitecore (IIS) - Razl, SPE Content Migrator

This blog post is simply about some basic troublshooting steps whenever we have integration with 3rd party modules which relies on custom APIs in Sitecore. 

First one in this discussion is Sitecore Razl. Sitecore Razl is a database comparison tool for the Sitecore CMS. It allows a developer to compare two different Sitecore databases and see the differences in each. A developer can then migrate changes from one database to the other. (ref)

Once the connection specific package is installed in a server, when trying to load the connection in the Razl tool, it would say it cannot connect. Same accessGuid is present in configuration and Razl Connection window. 

Second one is related to SPE Content Migrator. Script used to migrate content between Sitecore instances using Sitecore PowerShell Extensions.(ref).

In this case, even after enabling SPE Remoting and allowed role "sitecore\PowerShell Extensions Remoting" for authorization, migrator script always responded with the below error. 

Test Script:

Error:

Troubleshooting Steps: 

Both these errors have similar troubleshooting steps. Both these tools interact with Sitecore CM using custom APIs or Handlers. With the current client, there are many IIS rewrite rules added to support their business requirement or for security reasons in Content Management Server. 

• LowerCaseRule
• RemoveTrailingSlashRule
• Root Hit Force HTTPS Redirection
• Sitecore Login or Admin Force HTTPS Redirection
• Forbidden
• Redirect Error Page for a Business Requirement

First step - We enabled IIS logs for the CM site and also enabled all the available fields to log. Refer here to find the list of available fields in IIS Logs - https://www.finalanalytics.com/help/httplogbrowser/field-list.html. We tried multiple times from the tool (Razl or SPE Content Migrator) and we found the POST requests and it was redirected with 301 and a GET request was sent finally to Sitecore. Sitecore was not able to respond to the GET request as it was expecting a POST request from these tools. We tried to check the IIS rewrites one by one and found one of these rewrite rules are redirecting these requests. 

Razl: It was LowerCaseRule which redirected the Razl request. We added the negate rule to avoid Razl URLs. 

SPE Content Migrator: It was RemoveTrailingSlashRule rule which redirected all SPE migrator requests. We added the exception to prevent it. 

It is best to start with IIS first to understand the source of the problem. Happy Troubleshooting!.

Coveo for Sitecore - Endpoint Deprecation Issues? Upgrade the Coveo for Sitecore module

Even though we migrated Coveo endpoint to Organization specific endpoint, we still get email from Coveo that deprecated endpoints are being used. Looking at the traffic in Coveo administration, we were still getting much of the analytics traffic using deprecated endpoint. 

Coveo for Sitecore version: 5.0.1277.4 (Release date: October 23, 2023)

The latest Coveo for Sitecore module version (5.0.1368.1) has fixes for updating the analytics endpoint. In the release notes, we have quite a few Analytics related changes and proper analytics endpoints are being used with this version. 

WEB-6836: Fixed the content structure of the Coveo Page View Analytics request payload.

WEB-6922: Removed no longer required usage reporting.

WEB-6937: Fixed the analytics endpoint being used when you bypass the Coveo for Sitecore proxy.

WEB-6954: Simplified the way the Usage Analytics endpoint is composed, depending on the context.

Example: \Coveo\Hive\js\CoveoForSitecore.js

Once the module is upgraded, the traffic starts to drop in the deprecated endpoint and eventually leading to 100% in the recommended endpoint. 

Recent blog related to this change - https://www.nehemiahj.com/2024/11/coveo-for-sitecore-endpoint-deprecation.html

Coveo for Sitecore - Clickable URI when content is outside of all Site definitions (Shared or Global Pages)

Coveo for Sitecore generated Clickable URI at the time of indexing, but the site part of the clickable URI is recomputed automatically at query time. What if the content is outside of site definition and how Coveo for Sitecore will resolve Site and generate the Clickable URI - this blog gives an overview of how it is handled in Coveo.

During the indexing time, Coveo for Sitecore will try to resolve the site definition so that URIs can be generated. In the site definition, hostname attribute can have multiple domains or wildcards. If it has multiple values, then targetHostName is mandatory so that Coveo can use it to generate the URIs. 

This is the order by which Coveo for Sitecore will try to resolve the site using ResolveItemSiteProcessor.

1. First, it will eliminate all non-content sites - coveo_website, coveoanalytics, coveorest, coveoapi, shell, login, admin, service, modules_shell, modules_website, scheduler, system, publisher. 
2. It will concatenate rootPath and startItem of each content site definition and compare it with the Item path.
3. If the language attribute is available in Site definition, then it will be compared with the item language. 
4. If no site is selected with the previous steps, then it will choose the first site of the remainder content site.
5. If no site is resolved, then it will use Sitecore Setting 'serverUrl' value if available. 
6. If no site is selected in any of the steps, then default host of the Sitecore installation will be used. 

In our case, we have Global and Stores content page items which are not part of any of the site definition so the default host of the Sitecore installation was used to generate the URLs. Also, we have requirements where URL should be formatted in certain ways based on business requirements. 

The drawback of these items which do not have site definition is that Coveo for Sitecore will consider these items are simple indexable Sitecore Content Items rather than a page where HTML binary can be generated. When looking at the Content Browser, we can see huge number of Sitecore Items rather than HTML. 

Content Tree Structure:

In Coveo for Sitecore, there are many pipelines through which we can manipulate the content. In this case, the manipulation is required to set the context site and then use the custom ItemUrlBuilder to get the required URLs and it can be sent to index the pages. 

On a higher level, Coveo for Sitecore will run through these pipelines and processors.

For the item which are under a site context, clickableUri's are generated properly and Coveo for Sitecore will be able to send the request, generates the response HTML and send the indexable content to Coveo. For the items which are not under a Sitecore Site context, the default 'website' site definition will be used as the Site context. then the default host of the Sitecore installation will be used. 

If we are using <serverUrl>, we are tied to one URL per Sitecore instance. In case if we have multiple sites in a Sitecore instance, serverUrl concept will be defaulting to only one domain. 

There is a pipeline which helps Coveo for Sitecore to get the Site definition for the current item. Name is  coveoResolveItemSite. In this pipeline, out of the box processor ResolveItemSiteProcessor helps to find the site definition. Since these items are not under a valid site definition, it will resolve to default 'website' site definition. We added another processor in this pipeline to check if these items are Shared pages or the Store pages and then set the site name for the item. 

Processor to Set the Context Site: Site name is hard coded for this blog post.

Config Patch for Processor: 

Once the Context Site is set, the custom ItemUrlBuilder which is used in the website will kick in. That will help in generating the proper URL for these Global items. 

You may face an issue when rendering the results from Coveo. You may need to add custom processor in coveoProcessParsedRestResponse pipeline. You can do it only if the rendered URL in search results page has issues. 

Part 1: Coveo for Sitecore - ML Model - Product Recommendation (PR) - Introduction

Coveo ML for Commerce offers cloud-based AI-powered search and personalized recommendations to enhance customer engagement. In this blog series, I would like to give an overview of all the available ML Model configuration and strategies.

There are various commerce model types used in different applications.

• Product Recommendations (PR)
• Query Suggestions (QS)
• Automatic Relevance Tuning (ART)
• Dynamic Navigation Experience (DNE)

There are more advanced and personalized types. 

• Predictive Query Suggestions (PQS)
• Session-Based Product Recommendations (SBPR)
• Intent-Aware Product Ranking (IAPR)

In this series, we will discuss about Product Recommendations (PR) model which provides relevant product suggestion (for your Coveo-powered commerce implementation) to end users. This is heavily dependent on the Coveo Usage Analytics (Coveo UA) which tracks the end user's search event, product click events, add to cart events and purchase events. With this data, model gets trained continuously and starts returning relevant results to the end users. 

Coveo ML PR models offer strategies to adapt product recommendations to the evolving needs and preferences of customers throughout their shopping experience. Various strategy types for PR models are listed below. 

• User recommender
• Frequently bought together
• Frequently viewed together
• Cart recommender
• Popular items (viewed)
• Popular items (bought)

In the next article, we will discuss about the prerequisites of PR models.

Next Blog: Part 2: Coveo for Sitecore - ML Model - Product Recommendation (PR) - Prerequisites

Happy Searching and Happy Customer! 😊

Coveo for Sitecore - Endpoint Migration - Coveo UA Endpoint during Initialization

Coveo Usage Analytics (UA) Protocol is currently considered as Legacy. The newer Event Protocol for new Coveo for Commerce implementation is recommended. This blog article for the Sitecore client who uses Coveo UA Protocol to log commerce events. 

In order to send commerce events, it is important to initialize Coveo UA library. Please note that Coveo is deprecating legacy endpoints. You can read about here. In order to use Organization specific endpoints for this analytics events, it is important to provide the endpoint at this stage. 

Earlier, endpoint was not required during the initialization. As Organization specific endpoint is recommended, it is important to provide the endpoint URL during initialization. 

Happy Coding!

Coveo for Sitecore - Endpoint Deprecation & Migrate to Organization Specific Endpoint

When Coveo was launched, there were only a few API endpoints for search requests and analytics events. As Coveo has grown, the need for organization endpoint URLs has become increasingly important for several reasons:

• Clear Separation: Client-specific subdomains create clear boundaries between different clients
• This helps to isolate specific environment to an organization. 
• Independent Scaling: Ability for Coveo Infrastructure to scale specific client based on license.
• Coveo Infrastructure Security: 
• Isolation: Client-specific subdomains can isolate API traffic
• IP Whitelisting: By restricting access to specific IP addresses through the subdomain.
• Better Performance

As part of the endpoint deprecation, Coveo is moving towards the usage of organization specific endpoints for Search and Analytics events. 

<ORD_ID>.org.coveo.com - Search requests

<ORD_ID>.analytics.org.coveo.com - Usage Analytics logging

<ORD_ID>.admin.org.coveo.com - Organization administration

Currently, analytics.cloud.coveo.com is the designated domain for sending analytics events. Previously, platform.cloud.coveo.com served multiple purposes, including the Coveo Administrator console, search requests, and analytics. Coveo is deprecating the use of platform.cloud.coveo.com for analytics events. As of December 9th, 2024 June 3rd 2025 (updated on 11/14/2024), any analytics events sent to this domain will be ignored or result in errors. Along with this change, analytics.cloud.coveo.com is also marked as Suboptimal endpoint which encourages to use organization specific endpoint. 

How to check the current endpoint usage:

In order to check the current endpoint usage, you can go to Coveo Administration Console, Organization -> Settings. Under Organization tab, select Traffic. 

In the below screenshot, deprecated endpoint usage for last 7 days is listed. It also provides the recommended endpoint to use to avoid any interruption of service. 

How to update the endpoint in Sitecore:

• Open Sitecore CM Control Panel, and select Configuration Manager under Coveo Search. Coveo Command Center will be opened in a new tab. 
  
  
  
  
  
  
• In Coveo Command Center, click Log in.
• Log in to your Coveo organization.
• Click Apply and Restart.
• This action triggers Coveo for Sitecore to Switch to Organization Endpoints.
  
  
  

Under the hood:

In case, the following changes are updated in the Sitecore configuration. 

File Name: \App_Config\Include\Coveo.CloudPlatformClient.Custom.config

• Key: cloudPlatformUri
• Value: Changed from https://platform.cloud.coveo.com to https://<ORG_ID>.admin.org.coveo.com 
  
  
• Key: cloudGeneralPlatformUri (newly added)
• Value: https://platform.cloud.coveo.com 

File Name: \App_Config\Include\Coveo.SearchProvider.Rest.Custom.config

• Key: analyticsUri
• Value: Changed from https://analytics.cloud.coveo.com to https://<ORG_ID>.analytics.org.coveo.com 
  

• Key: searchApiUri
• Value: Changed from https://analytics.cloud.coveo.com to https://<ORG_ID>.org.coveo.com 
  

You can copy these two files from CM to each CD servers. 

Coveo for Sitecore - Exclude a site while resolving site for a Sitecore content item

When Sitecore tries to resolve the site for a content item while indexing, it will skip all the non-content sites like coveo_website, coveoanalytics, coveorest, coveoapi, shell, login, admin, service, modules_shell, modules_website, scheduler, system, publisher. 

In case if you have a content site which may interfere with site resolving functionality of Coveo, then you can use the attribute called 'mode' in the Site definition. This flag will exclude the site from Coveo site resolving feature. 

Sitecore Commerce - How do we trigger minion on-demand using postman?

While Composable Commerce is gaining traction, Sitecore Experience Commerce (XC) is still being used by many Sitecore customers. In this article, we will talk about Sitecore Experience Commerce Minions. Minions Role is an instance of the Commerce Engine that runs independently and supports asynchronous processing.

Based on business requirements, minions can be created and it can be scheduled to run at the appropriate time using cronjobs. Sometimes it needs to be triggered manually. We can use postman to initiate it. 

In order to run the minions, we need to get access token by hitting the /connect/token endpoint of Identity server. Using it, you can send a request to RunMinon() API with the Minion Full Name and you can see the status in the logs. 

Authentication

Get the access token by sending the request to token endpoint of ID server.

Trigger Minion

In order to trigger the minion, you need the minion full name. Once posted, it will start the job in the background and you can see the status in the log file. 

Hosting Sitecore XM Cloud Next.js frontend app in AWS Amplify

Next.js's creator, Vercel, is the preferred hosting solution. However, many other hosting providers have recently begun to support hosting frontend apps and various features of Next.js. One of them is Amazon Web Service (AWS) Amplify. 

AWS Amplify, backed by robust AWS infrastructure, is a hosting service provided by Amazon Web Service. You can host the frontend application like Next.js app or you can do full stack development using their Amplify Studio. AWS Amplify supports many features of Next.js but it lacks support on Edge API Routes (Edge middleware is not supported), On-Demand Incremental Static Regeneration (ISR), Internationalized (i18n) automatic locale detection, Next.js streaming, Running middleware on static assets and optimized images. You can refer this page for any updates on the Next.js feature support. 

To use AWS Amplify as the hosting provider, you can either use your existing subscription or sign up for trial AWS account which gives free access to some of their service for 12 months. 

Hosting XM Cloud Foundation Head

1. An XM Cloud project has been setup and Foundation template repository is cloned in your source control. 
2. Sign in to the AWS Management Console and open the Amplify console. 
   
   
   
   
3. In the AWS Amplify home page, click Get Started if you are using it for the first time.  
   
   
   
4. Click Get Started in Amplify Hosting. 
   
   
   
   
   
   
5. Select the source code location. In my case, it is GitHub. You will have to authorize Amplify to read/write your repository.  
   
   
   
   
   
   
6. Once the permission is set, you can choose the repository, branch and also check Connecting a monorepo and set the path as srx/sxastarter as the Next.js application is placed in that folder. 
   
   
   
   
   
   
7. When you click Next, you will see the build settings. You can see the build commands, app root folder and the artifacts base directory as well. 
   
   
   
   
   
   
8. Set the Sitecore related environment variables. You can get the SITECORE_EDGE_CONTEXT_ID from XM Cloud Deploy App --> Environment --> Details (Context ID (Live)).
   
   
   

   
   
   
9. Review the changes and click "Save and Deploy"
   
   
   
   
   
   
10. It will provision the environment, build and deploy it. You will get the AWS Amplify domain name at the end. https://*.amplifyapp.com/
    
    
    
    
    
    
    
    
    
    
    
    
11. When browsing the site https://main.d30qedhkgseqw4.amplifyapp.com/, you can see the content but it will be missing the styles. When checking the network tab, you can see the domain name for the static files as http://localhost:3000. 
    
    
    
    
    
12. You can go ahead and add an environment variable called PUBLIC_URL and add the newly generated amplifyapp.com URL including protocol. 
    
    
    
    
    
13. Now, your site will load perfectly with all the styles. https://main.d30qedhkgseqw4.amplifyapp.com/ 
    
    
    
    
    

PUBLIC_URL Environment Variable

The URLs which are generated for the static files depends on the domain name set by the hosting provider or the PUBLIC_URL Environment variable. PUBLIC_URL environment variable has higher precedence than any other environment variable. If it is not set, then it fall back to the hosting provider environment variable. 

Hosting providers like Vercel and Netlify provide the unique deployment URL in the system defined Environment variable by default. 

Vercel: VERCEL_URL.

Netlify: DEPLOY_URL

For AWS Amplify, as of now, the feature is not available and an issue is open in GitHub. The format of the Amplify URL is https://{branchName}.{AmplifyAppID}.amplifyapp.com. There are few ways to generate the URLs and I will discuss it in another post. 

To check how Sitecore Next.js SDK consume these URLs, please refer this Utils file in GitHub. 

Happy Learning!