getElementsByTagName - failing inconsistently

IamtheWalrus

I am using a Java application to read an XML file and process it. The application works in the majority of cases but not in some. I have an XML doc with a list of items. The app reads the doc and compiles a list of said items as follows:

NodeList responseItemNodeList = doc.getElementsByTagNameNS("ns", "myElement")

Sometimes a list is compiled and sometimes not. I have looked at different XML docs, ones that work and ones that don't and to me, they seem identical (apart from data).

Can anyone suggest why this is occasionally failing?

CreepingDeath

IamtheWalrus wrote: »

NodeList responseItemNodeList = doc.getElementsByTagNameNS("ns", "myElement")

If you replace "ns" with "*" it should return all the matching tags from all namespaces.

COYW

Could there be a file locking issue on the XML file? How are you accessing the data in the files? Are you catching any exception/s in your application on accessing the files or data within?

I experienced an issue like this before and it was due to the virus scanner constantly traversing the folder structure in which the XML files were located? 90% of the XML files would process fine but the rest would fail due to access errors.

IamtheWalrus

COYW wrote: »

Could there be a file locking issue on the XML file? How are you accessing the data in the files? Are you catching any exception/s in your application on accessing the files or data within?

I experienced an issue like this before and it was due to the virus scanner constantly traversing the folder structure in which the XML files were located? 90% of the XML files would process fine but the rest would fail due to access errors.

The xml is posted into my app and parsed there. I can see the XML that is to be processed in the logs. I add the xml elements to a list when I'm parsing the doc. The size of the list is zero, even though I can see the elements.

IamtheWalrus

CreepingDeath wrote: »

If you replace "ns" with "*" it should return all the matching tags from all namespaces.

I'll try this..

IamtheWalrus

I tried the "*" wildcard idea but I'm having the same issue,. Is it possible that the data is the issue, not the xml?

stevenmu

IamtheWalrus wrote: »

I tried the "*" wildcard idea but I'm having the same issue,. Is it possible that the data is the issue, not the xml?

It shouldn't happen if the XML is "well formed". But if it isn't then it's very possible. There might be some special character(s) in the data that are confusing your parser. Look for things like errant quote marks, apostrophes and ampersands in particular.

IamtheWalrus

stevenmu wrote: »

It shouldn't happen if the XML is "well formed". But if it isn't then it's very possible. There might be some special character(s) in the data that are confusing your parser. Look for things like errant quote marks, apostrophes and ampersands in particular.

Do you think switching to XPath is a good idea?

John_Mc

stevenmu wrote: »

It shouldn't happen if the XML is "well formed". But if it isn't then it's very possible. There might be some special character(s) in the data that are confusing your parser. Look for things like errant quote marks, apostrophes and ampersands in particular.

Yep - try opening the Xml files where it is happening in Internet Explorer. This will show you where the errors are if they exist.

COYW

IamtheWalrus wrote: »

The xml is posted into my app and parsed there. I can see the XML that is to be processed in the logs. I add the xml elements to a list when I'm parsing the doc. The size of the list is zero, even though I can see the elements.

OK, so you know that the xml is there for parsing. Do you have any exception handling around the parsing of the xml. If so, what exceptions are getting caught? Have you been able to recreate the issue in dev?

John_Mc wrote: »

Yep - try opening the Xml files where it is happening in Internet Explorer. This will show you where the errors are if they exist.

Yep, this will tell if there is an issue with the xml itself. Judging by the response above, the OP does not have xml files though. Personally, I always create xml files for audit reasons and I store them for 60 days. It has resolved many an issue with business users.

IamtheWalrus

COYW wrote: »

OK, so you know that the xml is there for parsing. Do you have any exception handling around the parsing of the xml. If so, what exceptions are getting caught? Have you been able to recreate the issue in dev?



Yep, this will tell if there is an issue with the xml itself. Judging by the response above, the OP does not have xml files though. Personally, I always create xml files for audit reasons and I store them for 60 days. It has resolved many an issue with business users.

I have the offending xml. I have opened it in IE and have seen no issues. It just opens. How do I tell if there are issues?

I have an example of xml that is parse-able and one that isn't. They look identical.

COYW

IamtheWalrus wrote: »

I have the offending xml. I have opened it in IE and have seen no issues. It just opens. How do I tell if there are issues?

I have an example of xml that is parse-able and one that isn't. They look identical.

There are no issues with the xml if it opens in the browser. You can take it that the xml is structured correctly.

Can you post the method/s in which you are parsing the xml?

As I said previously, is this error appearing across all environments, i.e. dev, uat & live? What are you doing within your application to handle the exception?

IamtheWalrus

COYW wrote: »

There are no issues with the xml if it opens in the browser. You can take it that the xml is structured correctly.

Can you post the method/s in which you are parsing the xml?

As I said previously, is this error appearing across all environments, i.e. dev, uat & live? What are you doing within your application to handle the exception?

I have put exception handling around the compile + evaluate parts of the Xpath attempt. Nothing has been caught.

The issue happens on my Test + staging environments. It's a long story but I cannot deploy the app locally. I have to make the change locally and deploy to Test. To test my changes, I have created a little parsing project locally to parse the 'unparse-able' xml. I am able to parse but when the code is added to my main app and deployed to Test, it fails. Exception handling removed below.

DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
        DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
        Document doc = docBuilder.parse (new File("sample.xml"));

        XPathFactory xFactory = XPathFactory.newInstance();
        XPath xpath = xFactory.newXPath();
        XPathExpression expr = null;

	expr = xpath.compile("/xx/yy/myElement");


        Object result = expr.evaluate(doc, XPathConstants.NODESET);

        NodeList list = (NodeList) result;

list is null.

onemorechance

If your xml file has a namespace you should set a namespace context for the xPath.

Here is an example for dealing with SOAP envelopes:

		 //set the namespace
		xPath.setNamespaceContext(new NamespaceContext() {

			@Override
			public Iterator getPrefixes(String namespaceURI) {
				// TODO Auto-generated method stub
				return null;
			}

			@Override
			public String getPrefix(String namespaceURI) {
				// TODO Auto-generated method stub
				return null;
			}

			@Override
			public String getNamespaceURI(String prefix) {
				if ("soapenv".equals(prefix)) {
					return "http://schemas.xmlsoap.org/soap/envelope/";
				}
				else if ("SOAP-ENV".equals(prefix)) {
					return "http://schemas.xmlsoap.org/soap/envelope/";
				}
				else if ("soap".equals(prefix)) {
					return "http://schemas.xmlsoap.org/soap/envelope/";
				}

				return null;

			}
		});

You should try to evaluate the xPath expression as a String first to test your xPath, get a count of nodes or a node value, if existing.

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
	<SOAP-ENV:Body>
		<ServiceName>
			<Date>2013-09-18</Date>
			<Id>1234567890</Id>
			<status>Success</status>
			...
		</ServiceName>
	</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

...
String xPathStatus = "SOAP-ENV:Envelope/SOAP-ENV:Body/ServiceName/status";
...
String expressionEvaluation = (String) expression.evaluate(doc, XPathConstants.STRING);
...

expressionEvaluation will return "Success" in the example above.

COYW

IamtheWalrus wrote: »

DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
        DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
        Document doc = docBuilder.parse (new File("sample.xml"));

        [B]// Check if doc is not null here and throw an exception if it is.[/B] 

        XPathFactory xFactory = XPathFactory.newInstance();
        XPath xpath = xFactory.newXPath();
        XPathExpression expr = null;

	expr = xpath.compile("/xx/yy/myElement");


        Object result = expr.evaluate(doc, XPathConstants.NODESET);

        NodeList list = (NodeList) result;

Put the exception handling around the whole block above. I would also add in a check to see if doc is not null, where I commented above. This will flag if the parsing is taking place but null is being returned. There is no point in evaluating the xpath and nodelist if doc is null.