Issues syncing Sales Order Lines with Dual Write

I am doing some experimenting with Synapse Links for Dataverse and to do that I need my FnO data in Dataverse. The way I do it, is using Dualwrite. When I try to sync Sales Order Lines V2 to salesorderdetails I get the following error:

Reason: Bad Request, Header x-ms-client-request-id f1004256-8a98-42fe-86a9-02b8e64a81a9, Produkten kan inte läggas till eftersom den inte är aktiv

(for those of you, not fluent in Swedish it says “The product cannot be added because it is not active)

This means that the product is synced but it is not active. To activate the product it needs to be published. To do this go through the following steps:

  1. In Sales open products
  2. Select a product and click Publish

The problem was I had a couple of thousand products. So I googled and found this forum thread helping me to write a workflow to automate it (this is way beyond my knowledge in CRM).

Links:
https://learn.microsoft.com/en-us/dynamics365/sales/publish-product-bundle-make-available-selling
Bulk publishing products – Dynamics 365 Sales Forum Community Forum

Data Events issues after refresh

In a previous article I wrote a bit about Data Events in Dynamics 365 for Finance and Operations. Data Events is a really simple way to create event-based integrations based on changes in the different data entities in FnO. The Data Events functionality is based on functionality in Power Platform. The setup requires the installation of the Finance and Operations Virtual Entity solution in the Power Platform environment connected to FnO. When you create a trigger Data Event it also created a virtual entity in DataVerse. This creates a couple of challenges when it comes to refreshing databases between environments.

For FnO
The settings for the endpoints created in FnO are partially stored in FnO and partially stored in an Azure Key Vault. The settings stored in FnO are amongst others Key Vault URL, App registration ID and Secret. To make sure that these settings do not get extracted from the environment (or accidentally moved to another environment) they are encrypted using an environment specific key and are thus not readable in the destination environment for the refresh. To restore the functionality of Data Events in the destination environment the endpoints need to be removed and recreated. After that has been done, we can re-activate the triggers.

Note that broken endpoints create an issue even if they are not being used. It seems like all endpoints are being validated when you try to create a new one which results in the creation failing.

For DataVerse/CRM
Since the functionality of Data Events is based on Virtual Entities created in DataVerse these will be overwritten when a refresh is done from one DataVerse environment to another. The error message you will get when the event is triggered is this:

Response status code does not indicate success: 404 ({“error”:{“code”:”0x80048d02″,”message”:”Virtual entity ‘mserp_vendvendorbankaccountentity’ not found for external entity VendVendorBankAccountEntity”}}).

(of course, with a different entity name based on your scenario)

The solution is to go to the Active Data Events tab in the Business Events workspace and remove and recreate each Data Event Trigger.

Note. You might have to wait a moment (1 minuter or so) before you recreate the trigger in order for everything to be properly cleaned in DataVerse.

Note: I have not been able to verify what happens if the identical triggers are set up in both source and destination environments. It might be that there are no issues or we might have the same issue. If anyone knows, please let me know 🙂

Lessons Learned: There has always been a lot to think about when you do refreshes… And DataVerse integration/DualWrite adds even more. There is a great article by Faisal Fareed here that detail steps that needs to be done for DualWrite integrated environments.

Links:
Not able to activate Data Events for entities – JohanPersson.nu
Microsoft Dynamics 365 and Power Platform Library: Steps to follow when refresh dual-write integrated environments (FO and CE) (daxture.blogspot.com)

DualWrite syncing empty lines on Initial Sync

I had this strange issue today…

One of my customers have set up DualWrite in their DEV environment and after some tweaking it worked OK. With that done they wanted to move the entire solution into TEST and verify that it worked. We packaged all the mappings into a Solution, exported it from DEV and moved it into TEST. We started enabling the Mappings for Legal Entity and a when we looked in CRM/CE we had a bunch of empty lines. We had the same number of lines but all of them were empty.

If we changed the data on one of the Legal Entities it synced over just fine but all the rest were still empty.

When we looked into the DMT files for the data project for the initial sync the files looked “good” to me… they contained data

|NAME|,|LEGALENTITYID|
|LegalEntity1|,|AAA|
|LegalEntity2|,|AAB|
|LegalEntity3|,|AAC|
|LegalEntity4|,|AAE|
|LegalEntity5|,|AAF|
|LegalEntity6|,|AAG|
|LegalEntity7|,|AAH|
|LegalEntity8|,|AAI|
|LegalEntity9|,|AAJ|
|LegalEntity10|,|AAK|
|LegalEntity11|,|AAL|
|Company accounts data|,|dat|

Some of you might already see the issue 🙂 (don’t spoil the surprise) When contacting Support they told me then there is a problem with the text qualifier in the file… the strings should be enclosed in ” instead of |

It turns out that someone had changed the default format CSV-Unicode in Data Management Framework to this:

I changed it back to this:

After cleaning out the records from CRM/CE and rerunning initial sync everything works again…

Warning:[DWCE0001] Export was skipped. Max lookup count supported in Initial Sync stage is 10. Current lookup count 11

Today I was setting up the Dual-Write sync for one of my customers and I bumped into this error message:

Warning:[DWCE0001] Export was skipped. Max lookup count supported in Initial Sync stage is 10. Current lookup count 11

The issue here is that there is a limit in DataVerse that (I would guess) for performance issues there is a hard limit on a maximum of lookup fields.

In our case we had these fields in the default mapping for Customer V3 -> Account using lookup:

transactioncurrencyid.isocurrencycode
msdyn_customergroupid.msdyn_groupid
msdyn_billingaccount.accountnumber
msdyn_paymentday.msdyn_name
msdyn_customerpaymentmethod.msdyn_name
msdyn_paymentschedule.msdyn_name
msdyn_paymentterm.msdyn_name
msdyn_vendor.msdyn_vendoraccountnumber
primarycontactid.msdyn_contactpersonid
msdyn_salestaxgroup.msdyn_name
msdyn_company.cdm_companycode

The workaround is to remove one of the problematic mappings, do the initial sync and add it back. Remember to take a screenshot of the mapping that you are removing so you can put it back exactly the same.