Could not find endpoint element with name... error when using a web part to connect to a WCf service

This is actually a more general WCf problem, but I decided to put it in the SharePoint section because it is more of a hidden problem in SarePoint development.

Typically, when you create a BizTalk WCF service and host it in IIS. Using it is as simple as consuming the service wsdl from your calling client via a service reference which will create entries in an app.config (or web.config) file corresponding to the needed / expected bindings, generating a proxy and using the service.

However, when you create a SharePoint web part for inclusion in a web page hosted in SharePoint server 2010 which in turn is used by to call the WCF service (in my case the WCf service was a BizTalk 2010 WCF service) you might get an error of the following type: Could not find endpoint element with name 'xxxx_xxx' and contract 'xxxx.xxxxx' in the ServiceModel client configuration section. This might be because no configuration file was found for your application, or because no endpoint element matching this name could be found in the client element

A long search on google actually returned a few potential )and sensible) solutions, none of which solved my problem.

1. One solution is suggested here http://www.roelvanlisdonk.nl/?p=735 which suggests making sure the name in the contract attribute of the endpoint is the same as you’re Microsoft Visual Studio Service Reference name. Basically, when you add the service reference to the WCF service in Visual Studio, make sure the first part of the 'contract name' in your app.config endpoint element configuration matches the service reference name specified
2. Another solution is suggested here http://www.vistax64.com/indigo/32159-location-name-config-file.html which suggests making sure that the fully qualified names are used in the app.config file for the contract attribute in the endpoint element. Meaning if the namespcae of your client applicaiton is smarties.mydev.callingclients, make sure the part of the endpoint element which spoecifies the contract is like: contract="smarties.mydev.callingclients.ServiceReference.Nameofcontractinservice"
3. Another solution here http://social.msdn.microsoft.com/forums/en-US/biztalkgeneral/thread/071cb8bd-4891-4e42-8f63-f18f74f47f3f/ suggests ensuring that your app.config informaiton is correctly copied into your application.exe.config (or in my case application.dll.config) file (where application refers to the name of your client application). This will usually be located in the same locaiton to which your .exe or .dll file is built, and should be more or less identical to the app.config file
4. The problem with my web part scenario was this: When you create a web part and add a service reference to it, it adds the configuraiton informaiton to the app.config file and (hopefully) the application.exe.config file. Unfortunately, when you ad the web part to a web page hosted in a SharePoint site, your client's app.config file is no longer used for the WCf call, as the hosting site's web.config file is the locaiton referenced for the configuration information. You need to copy your binding configuration information (<binding> element) and the endpoint confuguration information (<endpoint> element) into the relevant secitons of the <client> node within the System.Servicemodel section of the web.config file in the root location of the SharePoint site hosting the web page

After much thinking and brainstorming (since I am a BizTalk developer and not a SharePoint developer) and without much success searching on the Internet for my scenario, I solved the issue with the fourth step above

Object reference not set to an instance of an object error when consuming a web service exposed by the PeopleSoft Integration Broker

Object reference not set to an instance of an object error when consuming a web service exposed by the PeopleSoft Integration Broker

This error prevents adding the bindings from the web service to a Visual studio project even after using the svcutil tool to generate the wsdl and xsd files.

To fix this error, open the wsdl file in a text editor like notepad++

Open the wsdl types collection (<wsdl:types>)

Add a target namespace declaration to each imported schema in the colleciton:
e.g:
 <xsd:import schemaLocation="GetCompetenciesContainer.v1.xsd" namespace="http://www.mywsdlcompany.com/wcf/services/thiswsdlService/2010/01Container" />

becomes

<xsd:import schemaLocation="GetCompetenciesContainer.v1.xsd" namespace="http://www.mywsdlcompany.com/wcf/services/thiswsdlService/2010/01Container" targetNamespace="http://www.mywsdlcompany.com/wcf/services/thiswsdlService/2010/01Container"/>

Finally, add a target namespace declaration to the root schema node as well:

<xsd:schema elementFormDefault="qualified" xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.mywsdlcompany.com/wcf/services/thiswsdlService/2010/01Container">

Save the wsdl file and now use the Visual Studio tool with the wsdl and the downloaded xds files to generate the required bindings

BizTalk Development Errors: simple jargon-free problem and solution approach

------------------------------------------------------------------------------------

BizTalk Development Errors: simple jargon-free problem and solution approach

P: Visual Studio 2010 will not load BizTalk projects and BizTalk resources (like .btm BizTalk mapper files, .odx BizTalk orhcestration files etc)
S: You most likely installed Visual Studio after installing BizTalk. go to the 'Help --> About Microsoft Visual Studio' menu of Visual Studio and make sure that Microsoft BizTalk Server 2010 is in the list of installed products. If it is not, modify your BizTalk Server installation through the control panel (programs and features tab) and add the Developer Tools and SDK

P: When working with BizUnit or other xml-based tools, you get an error of the sort: the tag 'xxxx' does not exist in XML namespace 'clr-namespace:xxxx;assembly=xxxx'
S: First check the references in your project and make sure they are all fine. Secondly, make sure the reference to the namespace in xml is fully qualified (i.e. the full assembly name is used)

P: sqlservr.exe is using too much memory resources and affecting deployment of your biztalk applications due to out of memory errors

S: Fix SQL Server’s memory allocation. Go to SQL Server Management Studio, Right-Click on the Server, Properties à Memory. Set the maximum server memory (in MB) filed to a value your system can deal with, like 200MB for example

P: you get an ‘errors exist for one or more children’ error message when you’re trying to build a BizTalk orchestration project

S: this would usually be due to a problem in the scripts within an expression shape (the problematic one is usually identifiable by the red exclamation mark), or simply look through all your objects in the orchestration window to find any objects with the red exclamation mark. – This post http://stevestechnotes.blogspot.com/2010/07/biztalk-errors-exist-for-one-or-more.html also suggests some solutions

P: You get persistent ‘Schema referenced by Map "*****" has been deleted. The local, cached version of the BizTalk Server group configuration is out of date. You must refresh the BizTalk Server group configuration’ errors. Your list of BizTalk Applications isn’t showing in the Group. You try refreshing the BizTalk Group and it doesn’t solve the problem. You also try restarting host instance services and the [problem persists

S: See first part of the next solution below

P: When trying to use the baselangs.Any schema type for a map, you get 'Schema referenced by Map <mapname> has been deleted' error. Your BizTalk applications stop loading up in the Administration console.

S: If you’re only interested in getting your applications back in Administration console, open SQLserver, open the BizTalkMgmtDb, and open the dbo.bt_MapSepc table. Find the entry whose input (indoc_docspec_name) and output (outdoc_docspec_name)  schemas match your problem map and delete if from the table. This will remove the problematic map and allow the applications to load in BizTalk Administration Console.

However, if you’re hell bent on using your map and you wish to fix the problem, follow these steps. Delete your problematic application. Refresh the BizTalk Group. Add the Microsoft.XLANGS.BaseTypes dll to your BizTalk Group by adding it as an assembly to BizTalk Application 1. (Richt-click on BizTalk Application 1 à Add à BizTalk Assemblies à Browse to your BTS installation folder and pick the dll. Click on OK. Re-deploy your application

P: When trying to deploy you BizTalk solution, you get error: Parameter name: StrDBName:

S: The problem would usually have to do with the deployment properties in the btproj file (Properties -> Deployment -> Server) trying to deploy to a server to which you do not have access. It is probably set to another server other than your local sql server instance. Either change the server name to your local sql server instance, or if you are trying to deploy to a remote server, make sure the domain name is added to the server name (e.g. Sql server Devbbl02 on Domain btsdev.dev would be entered as Devbbl02.btsdev.dev)

P: when an xml message passes through a port, you get error: "An error occurred when parsing the incoming document: "Invalid Character in the given encoding"

S: The problem would usually have to do with improper or absent declaration in the xml file. BizTalk uses UTF-8 encoding by default in the disassemble and assemble stages of the receive and send pipelines. If as an example, a file in ANSI format containing foreign charcters (basically non-English characters - excuse my partialism) is passed through the receive port using the XMLReceive pipeline without the declaration (i.e. for the default U.S ANSI charset it would be "<?xml version="1.0" encoding="windows-1252"?> the pipeline simply would not be sure how to convert the byte representation of the foreign charcters (since different encodings use different byte representations for foreign characters). the pipeline would therefore not allow the xml file through and the error above would be thrown.

The easiest solution (which of course would almost never happen in the integration world) would be to have the declaration added to the xml file form source

Another solution is to use a custom pipeline component such as the one on this link. You would add it to the decode stage of the receive pipeline to let bizTalk know what encoding it will be dealing with. You then use that custom pipeline in the receive location

Finally, you could write a custom pipeline component to convert from ANSI to UTF-8 and use it in a similar way to the custom pipeline described above

P: after creating a Custom BizTalk Pipeline Component, the component does not show up in Visual Studio

S: To use a custom pipeline component in the Visual Studio design environment, you need three main steps. (a) write and build your component (b) GAC (Add the dll to the assembly) your pipeline component dll (c) add the component dll to the pipeline components directory in your BizTalk installation directory (typically c:\program files(or program files (x86) for 64-bit systems)\BizTalk Server 20_ _ (depending on your BTS version)\. After you have carried out all these steps right-click on the Visual Studio toolbox --> Choose Items --> BizTalk Pipeline Components.

If you do not see the components at this stage, close all open instances of Visual Studio, re-open Visual Studio, right-click on the Visual Stusio toolbox --> Choose Items --> BizTalk Pipeline Components. You should now see the component.

P: after creating a BizTalk Mapper Functoid, the functoid does not show up in Visual Studio

S: To use a custom mapper functoid in the Visual Studio design environment, you need three main steps. (a) write and build your functoid (b) GAC (Add the dll to the assembly) your functoid dll (c) add the functoid dll to the 'developer tools\mapper extensions\' directory in your BizTalk installation directory (typically c:\program files(or program files (x86) for 64-bit systems)\BizTalk Server 20_ _ (depending on your BTS version).

After you have carried out all these steps right-click on the Visual Studio toolbox --> Choose Items --> BizTalk Mapper Functoids. If you do not see the components at this stage, close all open instances of Visual Studio, re-open Visual Studio, right-click on the Visual Stusio toolbox --> Choose Items --> BizTalk Mapper Functoids. You should now see the functoid.

Creating an xml debugging file on the fly within development code

I sometimes use this technique for debugging during dev stages. Make sure you give the correct access to the folder in which you would like the xml file created

XmlTextWriter textWriter = new XmlTextWriter(@"C:\Testpipeline\myXmFile.xml", null);
                textWriter.WriteStartDocument();
                textWriter.WriteStartElement("Context");
                for (int i = 0; i < pInMsg.Context.CountProperties; i++)
                {
                    uvalue = pInMsg.Context.ReadAt(i, out sname, out snamespace).ToString();
                    //write all to new context except messagetype
                    if (sname != "MessageType")
                    {
                        outMsg.Context.Write(sname, snamespace, uvalue);
                        textWriter.WriteStartElement(sname, snamespace);
                        textWriter.WriteString(uvalue);
                        textWriter.WriteEndElement();
                    }
                    //promote all promoted
                    if (pInMsg.Context.IsPromoted(sname, snamespace))
                    {
                       outMsg.Context.Promote(sname, snamespace, nvalue);                    }                   
                }

                textWriter.WriteEndElement();
                textWriter.WriteEndDocument();
                textWriter.Close();

BizTalk 2010 message submission with transport-level security to SharePoint 2010 forms library without the WSS adapter

This was a solution I designed and built on one of my projects. The idea here was that the client had a process which needed to receive foreign exchange information from an external web service, have that information presented to certain management-level users in a readable, user-friendly fomat and added to a repository, and the information then needed to be read and approved as valid before the process would continue. One of the key design issues here was that for various reasons we could not use the SharePoint Adapter

I designed a BizTalk application which generated an xml file based on the foreign exchange currency information received from the external web service. An orchestration using a correlation set sent the information using WCF into a forms library hosted within SharePoint 2010, and then sent an email via SMTP on a dynamic port to certain management users with a link to the file. A loop ws created to continuously send the email after a certain number of minutes until the form was approved in SharePoint. They could then click on the link to open the document with an InfoPath form within SharePoint and approve the document. Upon approval, the approved xml was sent to a BizTalk WCF service hosted within IIS which (after confirming the approval based on certain business rules) sent a response to resume the same waiting BizTalk orchestration.

Whilst researching into this, I also came across R Seroter's wonderful blog post here which confirmed my design and helped speed up the development

I will try not to focus too much on the design of the BizTalk application itself as that is not the aim of this post. I will instead delve into the SharePoint and InfoPath aspects of the development more, and touch the BizTalk aspects architecturally

The steps are basically as follows:

1. Design an InfoPath 2010 form to approve the xml information
2. Create a SharePoint 2010 forms library based on the form design to accept the initial forms and store approved forms
3. Design a BizTalk orchestration which adds InfoPath processing instructions to an xml message in a message assignment shape (This is to make sure that once the xml messages are published, clicking on the xml documents will automatically open them in our InfoPath form).
4. The BizTalk orchestration sends the modified message to the SharePoint Forms library (through a WCf application which was created earlier) using a WCf-Custom adapter, as well as an email message to the management users (through the SMTP adapter) telling them the document is ready to be approved, and waits for a response (in my case, there is also a listen shape with a delay which sends a reminder email message to the management after a certain amount of time has elapsed to let them know the approval is holding things up)
5. The management users open the document using the InfoPath form and approve the document, thereby changing the status of the document to approved, and sending a confirmation to the web service
6. The web service sends a confirmation (approval) message directly to BizTalk, resuming the orchestration

PART 1 - Designing the InfoPath form.
(For this paticular implementation, make sure you have access to a SharePoint server, although with a little extra research, you can also implement similar but not exactly the same solutions without SP server (I used SharePoint 2010)

There are various ways to start off this part: for example you could start the form design from the schema based on the BizTalk message, or work backwards by basing the form design on the WCF service the form will be submitting to. I am not a SarePoint guru, but I found the second approach more foolproof, since you are sure the end result will submit successfully

1. Open InfoPath Designer (Start, All Programs, Microsoft Office, Microsoft InfoPath Designer)
2. Select 'New', and under 'Advanced Form Templates' choose the 'Web Service' option to create a form based on the web (WCF) service you wish to submit to
3. Click on 'Design Form' in the right-most panel
4. Make sure 'Submit data' is ticked on the next screen to indicate that your form will be submitting data to a service and click 'Next'
5. Enter the URL of the service WSDL (the service you wish to submit data to). If the WSDL address is correct, the wizard will expose the operations available in the service
6. Select the operation you wish to submit data to and click 'Next'
7. Enter a name for the data connection (e.g Submit To Approvals Service) and click 'Finish' (the form design view opens).
8. It is now important to remember that there are two services involved here. One is the WCF service we submit the form data to, which informs BizTalk that the form has been approved, and the other is the actual WCf service that BizTalk will call, which submits the forms to SharePoint in the first place. You should now see the schema structure expected by the WCF service we wish to submit data to in the right-side pane. Later you must ensure that your BizTalk application also uses this exact schema structure to create the xml message we want to send to SharePoint
9. Design the form interface by dragging the fields from the schema in the right-side panel. For more indepth information on this, you can try Kent Weare's post, which despite being based on an older version of SharePoint and InfoPath, is still very helpful. Also note that you can use the nifty repeating table to create the unbounded nodes from your schema quite easily. 
10. In the Controls section of the form menu, select and add any controls you wish to use on your form 
11. After all the form controls are in place, add a button to the form.
12. Right-click on the button and select 'Button Properties'
13. Select Submit as the button Action and type in a Label (e.g Approve). You can also do this using rules instead of the submit option
14. 
15. Click on 'Submit Options'
16. Click on the 'Advanced' button and fill in the page similarly to the diagram below (replace the data connection name with the name of your data connection that you created when making the form earlier, and fill in a label value to suit your preference).
17. Save the form as an xsn (form template). Note where the form template is saved to (you can use the save as dialog)
18. Click on 'File' on the menu bar and click on 'Publish'. Click on the 'SharePoint Server' button to indicate that you wish to publish the form to a SharePoint Server resource
19. Enter the address of the SharePoint site to which you wish to publish your forms. To understand this process better, you can read here or here or for some other ideas and information, you can read here
20. Select 'Form Library' and click Next. Some users of the form library might not have InfoPath installed. To enable such users to view and submit the forms in a browser, tick 'Enable this form to be submitted by a web browser'. If you don't have this option, go to Notes at the bottom of this post
21. Select the 'Create a new form library' option and click Next
22. Type in a descriptive name like 'ApproveRates' (make a note of this name for future reference), add a description (optional) and click Next
23. Add the xml/schema columns you wish to promote (make visible) in your forms library and click Next
24. Review the information, make a note of the Location and the Form Library name and click 'Publish'
25. Tick 'Open this form library' and click on the Close button
26. You should now see a preview of the new form library
27. My scenario was a SharePoint server which was part of a federated domain with the user profile service enabled. I now needed to set permissions for the BizTalk host instance to be able to add forms to the library, to avoid an error when trying to send the xml data through the SharePoint web services via BizTalk. I basically had to grant site permissions to the BizTalk user running the application pool under which the forms submission WCF service (the second service) would run. I granted permissions through the User Profile service (SharePoint Administration, 'Application Management', 'Manage Service Applications', 'User Profile Service Application'). I will not delve into this as I am not a SharePoint guru as I mentioned earlier. In a similar situation, I would assume you have a SharePoint administrator on hand to set the correct permissions
28. If your setup is simpler, you can also simply set permissions for the BizTalk user on the SharePoint site. To do this, browse to the actual site that will host your form library, click on Site Actions, Site Permissions, Grant permissions and add the BizTalk user with (at least) read and contribute permissions
29. To get the processing instructions, open the published form library by browsing to it (should be in the format http://your sharepoint server/your sharepoint site/your form library name)
30. Click on the 'Documents' tab, and click on 'Add Document' button as shown below

31. This will open up your form. Click on File and Save As to save the form. Ignore any validaiton warnings and click 'Yes' to save. Save the form as an InfoPath form (basically an xml file with InfoPath processing instructions, so that it automatically opens with InfoPath if InfoPath is installed on the user's computer) to a location you can remember
32. You have now finished the InfoPath forms design process

Notes

• If you want users of the form library to be able to submit the forms using Internet Explorer and you didn't have the tick box enabled at step 20 above, open a browser and browse to the sharepoint site in which you wish to host the form library (e.g. http:siteserver/site). You must be logged into SharePoint with an account that has enough priviledges to modify the site settings
• Click on Site Actions
• Click on Site Settings
• Under the Site Collection Administration heading, click on Site Collection Features
• Activate the SharePoint Server enterprise Site Collection Features
• Close the site
• Go back to step 19 above

Go on to Part 2 - The form submittal WCF service

Convert xml file to string inside BizTalk map using inline XSLT

I have a love for BizTalk maps because of their speed and the obvious load benefits when compared against orchestrations. I am therefore always pushing myself to find solutions that will avoid the use of orchestrations as much as possible when building applications. Inline XSLT is proven as one of the fastest methods for map conversions

One such endeavour led me to this solution. This solution allows you to convert a whole xml message into a string and feed it into one string node which can then be used in another map which (for whatever reason) may require a string message version of your original xml message. This could be done in an expression shape and message assignment shape in an orchestration, but as I said above, if you can avoid the load, why not?

1. Create a schema with a root node and a child field element node of string Data Type
2. Create a map which references your schema(s)
3. Add the xml schema to be converted as the source schema of your map
4. Add the new schema as the destination schema
5. Add a scripting functoid to the map
6. Connect the Root node of the source schema and the string data type (not the root) node of the source schema
7. Configure the scripting functoid (right-click, configure functoid script, Script functoid configuraiton) and choose Inline XSLT Call Ttemplate
8. Add the following script to the functoid: <xsl:template name="xml-to-string-called-template">
      <xsl:param name="param1" />
             <xsl:element name="ns0:NameofNodetoHoldString">
                   <xsl:text disable-output-escaping="yes">&lt;![CDATA[</xsl:text>
                   <xsl:call-template name="identity" />
                   <xsl:text disable-output-escaping="yes">]]&gt;</xsl:text>
              </xsl:element>
   </xsl:template>
   <xsl:template name="identity" match="@*|node()">
      <xsl:copy>
   <xsl:apply-templates select="@*|node()" />
      </xsl:copy>
    </xsl:template>
9. Test the map, and your whole source xml would have been nicely converted into a string and added  into the string node within the CDATA! section (which tells the processing receiver that everything within those should be treated as strings)

WCF / Web Service Serialization Error:Unable to generate a temporary class (result=1)

This was a funny one to fix:

Sometimes, when you consume a web service from within a wcf service or some other form of calling client application, trying to use the WSDL from the seb service gives the error described above.

This is due to serialization errors in the proxy code generation process. This is especially prevalent in cases where you are working with unbounded nodes from the xml files generated by the web service in BizTalk

According to Microsoft, "the Web Services Description Language tool (Wsdl.exe) generates proxy code for XML Web services. The generated types that are used in the proxy class are based on the contents of the WSDL document that describes the XML Web service. However, the generated types might not be what you want nor what you expect.

Wsdl.exe determines the best generated type to use for the objects that are specified in the service description. In some cases, the tool uses a least-common-denominator approach for casting objects to a type. For example, an ArrayList parameter in a WebMethod is described in WSDL as a XML Schema (XSD) sequence. When Wsdl.exe finds this array description in the service description, Wsdl.exe then generates a proxy class that uses an Object Array. You may prefer to work with an ArrayList, which was the original type that was used in the WebMethod. If you do not want to use the generated types, you can change the generated types to more desirable types. To get the appropriate object type, you can open the file that contains the generated proxy class, manually change the generated method parameter, and then return types to the appropriate object types"

Basically, what this means is if you add a service reference to a client applicaiton using the web service URI for example, you will generate some files from the wsdl of the web service.

If you browse to your application folder (the client application) on your computer, you should see a service references folder containing the folder for the particular service reference relating to the web service causing the error. In this folder, look for the reference.cs file and replace all the '[][]' with '[]'.

Save the file and close it. Next time you call the web service, you should not get the error above.