DualWrite syncing empty lines on Initial Sync

I had this strange issue today…

One of my customers have set up DualWrite in their DEV environment and after some tweaking it worked OK. With that done they wanted to move the entire solution into TEST and verify that it worked. We packaged all the mappings into a Solution, exported it from DEV and moved it into TEST. We started enabling the Mappings for Legal Entity and a when we looked in CRM/CE we had a bunch of empty lines. We had the same number of lines but all of them were empty.

If we changed the data on one of the Legal Entities it synced over just fine but all the rest were still empty.

When we looked into the DMT files for the data project for the initial sync the files looked “good” to me… they contained data

|NAME|,|LEGALENTITYID|
|LegalEntity1|,|AAA|
|LegalEntity2|,|AAB|
|LegalEntity3|,|AAC|
|LegalEntity4|,|AAE|
|LegalEntity5|,|AAF|
|LegalEntity6|,|AAG|
|LegalEntity7|,|AAH|
|LegalEntity8|,|AAI|
|LegalEntity9|,|AAJ|
|LegalEntity10|,|AAK|
|LegalEntity11|,|AAL|
|Company accounts data|,|dat|

Some of you might already see the issue 🙂 (don’t spoil the surprise) When contacting Support they told me then there is a problem with the text qualifier in the file… the strings should be enclosed in ” instead of |

It turns out that someone had changed the default format CSV-Unicode in Data Management Framework to this:

I changed it back to this:

After cleaning out the records from CRM/CE and rerunning initial sync everything works again…

Figuring out DataVerse and DualWrite in Dynamics 365 FnO

This is a (probably the first) post to try to sort out my experiences around setting up a DataVerse connection to a Finance and Operations environment and figuring out how this interacts with Power Platform Admin Center, LCS and DualWrite.

Background

DualWrite is Microsofts go-to solution for integrating Dynamics 365 CE and Dynamics 365 FnO. It uses Dataverse and PowerPlatform extensively which means that we are, in all essences, merging two separate products into one, which creates some challenges.

Since Microsoft is in the middle of a “Convergence” transition when it comes to managing these things I realize that this is a moving target at the moment, which is why I will need to come back to this this eventually.

This article will address some of the challenges that we have experienced, setting up DualWrite.

Since my primary focus is FnO I will start there.

There are a lot of clues to the fact that Microsoft sees PowerPlatform and DataVerse as an integral part of Dynamics 365 for Finance and Operations. The first one you will notice is that you get the option to create a Power Platform environment when you create a new FnO environment. Another lead is that none of the microservice add-ins that you can deploy from LCS are available to deploy it you have not connected your environment to Power Platform.

There are two different ways to create the DualWrite setup. Setting up a new, empty Environment when deploying a new FnO environment or linking your FnO environment to an existing CRM/CE environment. Please remember that, if you have an existing CRM environment with existing customizations (of a highly customized FnO environment) you should probably think about setting up a proof of concept to evaluate how to handle customizations. Keep in mind that the out-of-box mappings for DualWrite are created for vanilla environments.

Initial Setup

When setting up a new Finance and Operations environment you get the option of also setting up a new connected DataVerse environment. You will not get the option to connect an existing environment. You are able to opt out of this setup at the time of deployment if you want.

Regardless of what you choose the environment will be created and connected from the Power Platform side. On the LCS side there is no indication of any DataVerse environment.

Connecting to PowerPlatform

NOTE: This decision is IRREVERSIBLE. Once you have linked your FnO environment to a Power Platform environment there is no supported way to unlink it.

Once the environment is set up LCS offers an option to set up the DataVerse Connection. You can use the one provisioned for you, if you are not using CRM or if you are not planning to use DualWrite to interface with CRM, or you can link it to your existing Dynamics 365 for Sales (CRM) environment. Even though the connection is done to your existing/live CRM environment the operation should be safe since the Power Platform are being deployed to another “partition” of the environment. I know, the message in the upper right corner looks a bit scarry…

This operation only enabled the install of add-ins, DualWrite still needs to be set up from within FnO when you are ready for it.

Lessons Learned

Since Microsoft is currently moving the management experience of Dynamics 365 for Finance and Operations environments to the Power Platform Admin Center, all of this is a changing scenario and I think what we are seeing is a transition to what is about to come.

Key Take-Aways

  • Do a gradual rollout, starting with some entities
  • If there is data that does not need to be synchronized, a different solution such as virtual entities or PowerApps could be an idea
  • Do a proof-of-concept to validate the setup

Links:
Enable integration during environment deployment
Microsoft Power Platform integration with Finance and Operations apps – Finance & Operations | Dynamics 365 | Microsoft Docs

Enable vs Enabled Features

I have learned that it is important to read the fine print… otherwise you will miss things. This happened to me and a colleague the other day.

In Dynamics 365 for Finance and Operations, Microsoft add new features all the time. These are controlled in Feature Management workspace where you can also read up on the new features to understand important impact.

A cool feature is that there is a Data Entity in the Data Management Framework which exports and imports the feature set for a given environment and enables moving feature settings from one environment to another making easier to manage the lifecycle of your features and sync them with for instance releases

Now comes the part that I completely missed (which in hindsight is quite obvious):

There is a column called Enable Date, which I thought meant “The date the feature was enabled”, what it actually means “The date the feature is enabled”… notice the subtle difference?? I did not 🙁

What the column actually does is to set a schedule for when the feature will be enable. When you use DMF to import a list of feature settings with this field set means that you will schedule the enablement of the feature. It is a great feature but might cause some issues if you are not aware of it. Especially for features that can not be turned off.

Remember: Read and understand the fine print

What happened to batch groups??

I need to confess something… Sometime I do not read all of the release notes as thoroughly as I maybe should. This was made clear to me the other day when I tried to set up a batch job and make sure it was executing in a new Dynamics 365 for Finance and Operations Production environment. Or maybe I read it, but did not really understand it…

Before, you were able to create a batch group and add servers to it. This was used in for instance AX 2012 or D365FO to control execution and divide resources in an optimal way between batch jobs. Especially when you were dealing with large, time consuming, batches that you needed to ensure ran correctly while not starving smaller (time critical) jobs of resources

Starting with version 10.0.29 the default behaviour is that all Batch Servers are assigned to all Batch Groups. Each Batch Group has a scheduling priority set to either Low, Normal, High, Critical or Reserved Capacity and the batch jobs are then assigned to the Batch Group meaning that you will (almost) always have load balancing over all servers. The exception to the rule is if you choose Reserved Capacity.

Reserved Capacity means that you on an environment level can exclude a percentage of the total batch capacity (aka Batch Threads). The setup is done in System Parameters – Batch global settings. The default setting is “No reserved capacity”, meaning that all batch servers threads are available to the load-balancing. You are able to change this to Low, Medium or High (10, 15 or 25 percent) which will then exclude batch capacity from the pool. Worth knowing is that when no batch jobs with Reserve Capacity are executing the reserved batch threads will be sitting idle.


Links
https://docs.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/sysadmin/priority-based-batch-scheduling

Missing MR report in new legal entity

Yesterday a colleague contacted me about missing all of the reports in Financial Reporting (previously Management Reporting). After looking around a bit and doing the mandatory reset of the DataMart (which did not help) I found this article mentioning the minimum steps needed in a new legal entity for it to appear in the Financial Reporting UI.

Link
https://docs.microsoft.com/dynamics365/finance/general-ledger/financial-reporting-getting-started

Microsoft 365 add-ins blocked

Today one of ´my colleagues had an issue on her customer laptop. The issue was that she was not able to use the “Open in Excel” funcitonality in Dynamics 365 for Operations and Supply Chain. Apparently someone had blocked the Office addin that was required to make it work.

There are 2 options to handle this… the easiest (and less secure) option is to allow the user to run and install Office Addins. The setting resides in the Office Admin Center and requires an Admin to change it. If some one has disabled it, go the Office Admin Portal – Setting – Org Setting – User owned apps and services and enable “Let users access the Office Store”

The second option is to provision the specific apps to the users that need them. This is also done by an admin in the Office Admin Portal.

In the Office Admin Portal, go to Settings, Integrated Apps and click Add-ins

Click Deploy Add-in

Click Next

Click Choose from the Store

Seach for Dynamics and click Add next to the Microsoft Dynamics Office Add-in

Click Continue

Now you will get the option to set which users will get the add.in deployed. You have the options of Everyone, an Azure AD Group (preferred) and just you. The last option is used for testing and you will later in the wizard get the option to change.

Note: If you are logged in ad Global Admin, that is the user that will get the add-in. You should preferably not use your “regular user” to manage your organizations infrastructure. If you are curious, google Tiering…

Verify that your deployment was successful

You are now done. If everything works, now you get another option to assign which users should get the app.

You can click on the add-in and adjust the deployment scope.

That is it… Good Luck

Links
Prevent add-in downloads by turning off the Office Store across all clients (Except Outlook)
Deploy add-ins in the Microsoft 365 admin center

Cost for Azure Hosted Build Pipelines

In Dynamics 365 For Finance and Supply Chain we rely heavily on Azure DevOps for managing aspects of our projects, especially development and deploying packages.

Today a customer told me that they hit the roof on the free/included alotment for parallell build in Azure Hosted Pipelines. They wanted to understand what it had been used for. I found a very handy new preview feature called Historical Views for Pipelines.

To turn it on you you click on the settings icon in the top right corner and click Preview Features. The feature can be turned on for single users or entire organizations.

This feature will help you understand your usage.

The first thing you need to in order to purchase more parallel jobs is to set up billing for your Azure DevOps organization. Go to Organization Setting – Billing. The billing options are the same ones that you are using for “regular” Azure.

After you have set up billing, go to Organization Settings – Parallel Jobs and select Purchase Parallel jobs.

The pricing for additional paralell jobs are $40/month which gives you 1 paralell job with unlimited minutes.

Links
Historical graph for agent pools – Azure Pipelines | Microsoft Docs
Microsoft-hosted agents for Azure Pipelines – Azure Pipelines | Microsoft Docs
Configure and pay for parallel jobs – Azure DevOps | Microsoft Docs
Manage preview features – Azure DevOps Services | Microsoft Docs
Set up billing for your organization – Azure DevOps Services | Microsoft Docs
Azure DevOps Services Pricing | Microsoft Azure

Uppdating the e-Commerce solution in Dynamics 365 FO

A while ago Microsoft launched an e-Commerce platform for Dynamics 365 and they keep release updated versions in line with the OneVersion vision. Since I have not had the opportunity to work a lot with the solution I thought I would write down some learnings… if nothing else for myself. In this article I will look at how to update.

The first step is getting the update from Microsoft. As usual we start in the Asset Library. From the D365FO environment page click Manage to go to the Management console for e-Commerce.

Select the e-Commerce tab

Go to the Assett Libabry

Go to e-Commerce Packages and click Import

On the Import screen, note that all of the packages have notification marking the mail FnO version. Choose the latest package with your FnO version and click Pick.

When you are back in LCS, select the imported package and click Release Candidate

Once the package is set to Release Candidate it will show up in package deploy of the e-Commerce Management. To deploy it, click Apply Extension

DMF not creating new project

I had a question today regarding a user that, when she tried to create a new Import project in the Data Management Framework, she ended up in an old existing project.

We tried to clean out usage data and also restarted the server and browser to clear potential cache issues.

Another colleague told me that the issue is relates to saved views… and low an behold, when changing the standard view to be the default it all worked again.

Warning:[DWCE0001] Export was skipped. Max lookup count supported in Initial Sync stage is 10. Current lookup count 11

Today I was setting up the Dual-Write sync for one of my customers and I bumped into this error message:

Warning:[DWCE0001] Export was skipped. Max lookup count supported in Initial Sync stage is 10. Current lookup count 11

The issue here is that there is a limit in DataVerse that (I would guess) for performance issues there is a hard limit on a maximum of lookup fields.

In our case we had these fields in the default mapping for Customer V3 -> Account using lookup:

transactioncurrencyid.isocurrencycode
msdyn_customergroupid.msdyn_groupid
msdyn_billingaccount.accountnumber
msdyn_paymentday.msdyn_name
msdyn_customerpaymentmethod.msdyn_name
msdyn_paymentschedule.msdyn_name
msdyn_paymentterm.msdyn_name
msdyn_vendor.msdyn_vendoraccountnumber
primarycontactid.msdyn_contactpersonid
msdyn_salestaxgroup.msdyn_name
msdyn_company.cdm_companycode

The workaround is to remove one of the problematic mappings, do the initial sync and add it back. Remember to take a screenshot of the mapping that you are removing so you can put it back exactly the same.