Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Jan 27, 2011 @ 14:22
    Jeroen Breuer
    0

    xpath slow on large xml

    Hello,

    I've got a pretty big xml file which is loaded with umbraco.library:GetXmlDocumentByUrl. I use xpath to get an element which is pretty deep in the tree. Here is my xpath:

    $index/index/home/chambers/chamber/subitems/subitem[@id=$id]

    This seems pretty slow. Is there a way to optimize my code? Can the xpath be faster some other way or can I somehow fragment the xml file more for better performance?

    Jeroen

  • Douglas Robar 3570 posts 4711 karma points MVP ∞ admin c-trib
    Jan 27, 2011 @ 14:36
    Douglas Robar
    1

    That xpath looks good in that you aren't using '//' or 'descendant-or-self' but are instead specifying exactly how to get to the subitem you want. 

    It may be that you have a LOT of subitem entries in the XML?

    But more likely, the GetXmlDocumentByUrl() is sluggish retrieving a large XML file due to latency on the http request and bandwidth to return the xml itself. If you can, use the optional timeout parameter with GetXmlDocumentByUrl() to reduce the frequency of the request for the xml file itself. And macro caching will also be your friend. You might also want to look at Darren's FeedCache package, which is excellent.

    Let us know what you find out.

    cheers,
    doug.

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Jan 27, 2011 @ 15:20
    Jeroen Breuer
    0

    Hi Doug,

    I've been debugging the code and getting the xml with GetXmlDocumentByUrl works fine. The slow part is where I do my xpath. There are a lot of subitem entries indeed. I was hoping that I could improve my xpath somehow. Maybe narrow down in which element I need to search. Thanks for helping.

    Jeroen

  • Gerty Engrie 130 posts 490 karma points c-trib
    Jan 27, 2011 @ 16:22
    Gerty Engrie
    1

    You can also pass a caching parameter to the GetXmlDocumentByUrl function :-)

  • Chriztian Steinmeier 2800 posts 8790 karma points MVP 8x admin c-trib
    Jan 27, 2011 @ 16:32
    Chriztian Steinmeier
    2

     

    Hi Jeroen,

    If each step has a lot of items it might help to hand-pick the one you're after (if at all possible) e.g.

    $index/index[1]/chambers[1]/etc...

    - though that's probably not possible for you.

    You should try using a key - tricky part is it only works on the document containing the current node (not $currentPage), so the trick is to change context with the for-each instruction - something like this:

    <!-- Before first template -->
    <xsl:key name="subitems-by-id" match="subitem" use="@id" />
    
    <!-- Somewhere after fetching XML doc into $index -->
    
    <xsl:for-each select="$index">
        <xsl:apply-templates select="key('subitems-by-id', $id)" />
    </xsl:for-each>
    
    
    <xsl:template match="subitem">
        <!-- do stuff -->
    </xsl:template>

    /Chriztian

     

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Jan 27, 2011 @ 17:00
    Jeroen Breuer
    0

    Hi Chriztian,

    I'll try this and see if it works. Is this the best way if I want to fetch only 1 item? On twitter some also said I could use this:

    $index//subitem[@id=$id]

    Somehow I think your key technique is better :). I'll let you know when I got it working. Thanks!

    Jeroen

     

  • Douglas Robar 3570 posts 4711 karma points MVP ∞ admin c-trib
    Jan 27, 2011 @ 17:04
    Douglas Robar
    0

    Using "//" will definitely be slower. Much slower.

    cheers,
    doug.

  • Chriztian Steinmeier 2800 posts 8790 karma points MVP 8x admin c-trib
    Jan 27, 2011 @ 17:11
    Chriztian Steinmeier
    0

    If you're only fetching a single item, the key might not help (curious to know though).

    The // thing will likely be the slowest possible solution ;)

    /Chriztian

  • Douglas Robar 3570 posts 4711 karma points MVP ∞ admin c-trib
    Jan 27, 2011 @ 17:14
    Douglas Robar
    1

    After a bit more reading on xsl:key and key(), I came across this tidbit that helped explain...

    "The xsl:key element alerts the XSLT processor to create an indexed data structure for the key expressions ahead of time. This indexed data structure will be used by the keyfunction."

    So even though creating the index takes a bit of time you could easily gain it back when you search for an entry with the key() function. The more nodes you're chewing through the more gain you'll get.

    Though I've used keys in the past I learn something new every day!

    cheers,
    doug.

     

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Feb 03, 2011 @ 13:28
    Jeroen Breuer
    0

    This is how my code is now:

    <xsl:key name="subitems-by-id" match="subitem" use="@id" />
    
    <xsl:variable name="tempSubitem">
        <xsl:for-each select="$index">
          <xsl:copy-of select="key('subitems-by-id', $currentPage/@id)" />
        </xsl:for-each>
    </xsl:variable>
    
    <xsl:variable name="subitem" select="msxml:node-set($tempSubitem)/*" />
    

    Don't think there is a big performance boost, but I think it's faster :).

    Jeroen

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Feb 03, 2011 @ 14:00
    Jeroen Breuer
    0

    Hmm I'm having another problem now:

    First I just to got the subItem and than it's parent like this: 

    <xsl:variable name="subitem" select="$index/index/home/chambers/chamber/subitems/subitem[@id=$currentPage/@id]" />
    
    <xsl:variable name="chamber" select="$subitem/ancestor::chamber" />
    

    Now I've got this:

    <!--Fetch the subitem from the index xml. This could also be done using an xpath, but this is faster.-->
      <xsl:variable name="tempSubitem">
        <xsl:for-each select="$index">
          <xsl:copy-of select="key('subitems-by-id', $currentPage/@id)" />
        </xsl:for-each>
      </xsl:variable>
    
      <!--Convert the subitem into the right format to work with.-->
      <xsl:variable name="subitem" select="msxml:node-set($tempSubitem)/*" />
    
      <xsl:variable name="chamber" select="$subitem/ancestor::chamber" />

    In the first example I got the Chamber element back, but in the second I don't. Is there a way to get the parent of a node which is fetched with a key?

    Jeroen

     

  • Chriztian Steinmeier 2800 posts 8790 karma points MVP 8x admin c-trib
    Feb 03, 2011 @ 21:20
    Chriztian Steinmeier
    1

    Hi Jeroen,

    The reason you can't access the chamber node is because your subitem is nothing but an *evil* clone (you used copy-of to create it) that doesn't belong to the same tree as $currentPage (and thus, the chamber node).

    If you do the template thing I did, you'll have access to the entire tree from within that template, and you can create a variable for the chamber node as you did before, e.g.:

    <xsl:key name="subitems-by-id" match="subitem" use="@id" />
    
    <!-- Somewhere after fetching XML doc into $index -->
    <xsl:for-each select="$index">
        <xsl:apply-templates select="key('subitems-by-id', $id)" />
    </xsl:for-each>
    
    <xsl:template match="subitem">
        <xsl:variable name="chamber" select="ancestor::chamber" />
        <!-- This might be quicker if chamber is always 2 steps away: -->
        <!-- <xsl:variable name="chamber" select="../.." /> -->
    
        <!-- Do stuff with $chamber -->
    </xsl:template>
    

    /Chriztian

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Feb 03, 2011 @ 21:25
    Jeroen Breuer
    0

    Hi Chriztian,

    Thanks for the tip! Only problem is that I want to use the chamber and subitem variable inside multiple templates. In my example I set the variable before any template is called. Can I somehow do your sample without using templates (a bit like my sample)?

    Jeroen 

  • Chriztian Steinmeier 2800 posts 8790 karma points MVP 8x admin c-trib
    Feb 03, 2011 @ 21:54
    Chriztian Steinmeier
    1

    Hi Jeroen,

    OK, well - I'd suggest you send the subitem variable along when you're calling other templates, e.g.:

    <!-- The subtem template from before -->
    <xsl:template match="subitem">
        <!-- Start your errands here -->
        <xsl:call-template name="runErrands">
            <xsl:with-param name="subitem" select="." />
            <xsl:with-param name="chamber" select="../../" />
        </xsl:call-template>
    </xsl:template>
    
    <xsl:template name="runErrands">
        <xsl:param name="subitem" />
        <xsl:param name="chamber" />
    
        <!-- Do stuff -->
    </xsl:template>
    

    You can of course keep calling others, sending the variable/parameters along everytime.

    Hope that's useful, then :-) 

    /Chriztian

  • Jeroen Breuer 4908 posts 12265 karma points MVP 5x admin c-trib
    Feb 07, 2011 @ 17:11
    Jeroen Breuer
    0

    Hi Chriztian,

    I've applied your sample and it works like a charm :). Thanks for helping!

    Jeroen

Please Sign in or register to post replies

Write your reply to:

Draft