Choosing a Subset of Records

[phillip.isley]

I am attempting to use the "startFrom" and "maximumNum" elements to query a large amount of data in batches. It does not appear to be working correctly, but is most likely a mistake on my part.

I am using the information found here https://developer.cisco.com/media/webex-xml-api/26GlobalRequestElementsforListControl.html#ChoosingASubsetofRecords to control what data I would like back.

The first query is successful and when looking at the "matchingRecords" tells me:

startFrom = 1

maxNum = 500 (ceiling per request based on the doc)

totalRecords = 6471

For the next query, I modify the startFrom variable to 501 (since I already have the first group of 500) and post the request. The response status code is OK, however the returned value is just empty. Should I not be able to get all 6471 records by incrementing the startFrom element to collect 500 at a time?

Am I using the startFrom, maxNum, and totalRecords elements incorrectly?

[phillip.isley]

After further investigation, my code is working as it should and there is simply an empty record. This shouldn't be something that happens, correct? I found this by testing various startFrom and setting maxNum to 1 so that the request would only ask for one entry. I found that startFrom = 520 returns a message response of OK, but the result is completely empty. This was causing any other request range that included 520 to "blow up" upon trying to deserialize it.

If a record exist, it should not be empty, correct? If this is something that is unavoidable, the only way I know to check for it is by requesting one record at a time and testing it's value. This would be fairly resource heavy since it's an individual POST for each record. Please tell me this isn't the only way to fix this.

[ryanhunt]

What API Call are you using and what's your WebEx site? I will see if I can find out whats wrong with that record and blowing it up.

[phillip.isley]

<?xml version="1.0" encoding="utf-8"?>
<serv:message
  xmlns:history="http://www.webex.com/schemas/2002/06/service/history"
  xmlns:com="http://www.webex.com/schemas/2002/06/common"
  xmlns:att="http://www.webex.com/schemas/2002/06/service/attendee"
  xmlns:meet="http://www.webex.com/schemas/2002/06/service/meeting"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:use="http://www.webex.com/schemas/2002/06/service/user"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:ep="http://www.webex.com/schemas/2002/06/service/ep"
  xmlns:serv="http://www.webex.com/schemas/2002/06/service">
  <serv:header>
       <serv:securityContext>
            <com:webExID>sas.admin</com:webExID>
            <com:password>XXXXX</com:password>
            <com:siteID>852782</com:siteID>
            <com:partnerID>guXv7Mx9mVjgvcsJzdh4_g</com:partnerID>
       </serv:securityContext>
  </serv:header>
  <serv:body>
       <serv:bodyContent xsi:type="ep:lstRecording">
            <ep:listControl>
                 <ep:startFrom>544</ep:startFrom>
                 <ep:maximumNum>1</ep:maximumNum>
            </ep:listControl>
            <ep:createTimeScope>
                 <ep:createTimeStart>01/01/2010 00:00:01</ep:createTimeStart>
                 <ep:createTimeEnd>02/05/2016 14:18:04</ep:createTimeEnd>
            </ep:createTimeScope>
            <ep:returnSessionDetails>true</ep:returnSessionDetails>
       </serv:bodyContent>
  </serv:body>
</serv:message>

https://sas.webex.com/

Currently, it's record 544  with the date range found in the XML. It is moving though since this is a production site and new records are being added regularly. I know my code is working correctly as a whole since I pulled 35k records from our old WebEx site successfully. This is a simply a matter of, why is this record empty instead of just not existing at all?

[ryanhunt]

After speaking with our engineering team, it looks like one of your users: Antonio Gianni has a recording with an invalid XML character in the title, can you reach out to this user and get them to fix this recording name? It appears the problem is with a character between 'IT Enterprise Solutions' and 'Scheduling'

[phillip.isley]

Thanks for the explanation Ryan. I will check with my user and test after a change. Do you have a list of invalid characters? Also, do you know why there is nothing in place to prevent users from entering an invalid character when saving a recording? This creates a big problem for the XML API since it returns an empty/null response. Would it be possible to open a bug report for this problem?

[ryanhunt]

The problematic character is likely an em dash, or long hyphen.. The likely fix is to escape it into an XML Unicode mdash, I am still working w/engineering on a solution..

As a work around in the meantime, fixing this record manually should address the immediate issue.

[phillip.isley]

Thanks again Ryan for all the help. I'm not sure what the actual character was, but I had my user modify the file name and it is no longer causing a problem. If possible, please keep me informed of when a final solution is implemented.

[phillip.isley]

Hey Ryan,

Were there ever any changes made to address the issue of invalid XML characters used in Meeting/Recording names? I am running into this problem again and looking for a final solution. I haven't found a way to handle it from my side as the WebEx API doesn't return any data if an invalid character is present. I believe this is something that will have to be addressed from the WebEx side. The most likely solution is to escape all invalid XML characters before sending the response.

If I am missing something that I can do from my side, please let me know. I have had to revert back to processing the API requests in batches of 1 instead of 500 to ensure I don't exclude large chunks of data. As you can imagine, this greatly increases the time it takes for the full job to run. Please let me know if there is anything I can do to help be it testing, providing code, etc.

Thanks,

Phillip