SAML2, Certificate Authorization and Web Services to HIE

We are trying to connect to an external HIE (Health Information Exchange) using web services. They are using SAML token and certificate to validate. The HIE have installed our certificate and we are using the below code to call the service end point of HIE. . Whenever I try the below code segment, we get either Authorization error or 400 bad request? Any help on if we need to do something else?

            string endPoint = @"https://www.test.com";
        string key2 = @"Cert.pfx";

        string filepath = @"SOAPrequest.xml";
        TextReader xmlRead = new StreamReader(filepath);
        string payLoad = xmlRead.ReadToEnd();
        xmlRead.Close();

        // Cert Issues by Our Organization
            X509Certificate2 cert2 = new X509Certificate2(key2, @"Password");
            var assert = new Saml2Assertion(new Saml2NameIdentifier(@"ourOrg.com"));
            assert.SigningCredentials = new X509SigningCredentials(cert2);
            assert.Subject = new Saml2Subject();
            assert.Subject.NameId = new Saml2NameIdentifier(@"Our Organization") { Value = "ourorg" };

            assert.Id = new Saml2Id();
            assert.IssueInstant = DateTime.Now;



            Saml2Attribute org = new Saml2Attribute("urn:oasis:names:tc:xspa:1.0:subject:organization");
            // Adding atrribute values

            var item = new Saml2AttributeStatement();
            item.Attributes.Add(org);
            assert.Statements
                  .Add(item);

            // Write the token to XML
            Saml2SecurityToken saml2 = new Saml2SecurityToken(assert);
            StringBuilder bld = new StringBuilder();
            XmlWriter wr = XmlWriter.Create(bld);
            WSSecurityTokenSerializer serializer = new System.ServiceModel.Security.WSSecurityTokenSerializer();
            Saml2SecurityTokenHandler hdlr = new Saml2SecurityTokenHandler();
            hdlr.WriteToken(wr, saml2);

            // Add the PFX Certificate
            ServicePointManager.ServerCertificateValidationCallback = (a, b, c, d) => true;
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(endPoint);
            req.AllowAutoRedirect = true;
            req.ClientCertificates.Add(cert2);

            // Add SAML
            req.Headers[HttpRequestHeader.Authorization] = bld.ToString();
            req.Method = "POST";
            req.ContentType = "text/xml;encoding='utf-8'";
            WebResponse resp = req.GetResponse();

TIA Regards Shankara