topic Re: Choosing a Subset of Records in Cloud Collaboration

Choosing a Subset of Records

phillip.isley — Fri, 29 Jul 2016 15:39:24 GMT

I am attempting to use the "startFrom" and "maximumNum" elements to query a large amount of data in batches. It does not appear to be working correctly, but is most likely a mistake on my part.

I am using the information found here https://developer.cisco.com/media/webex-xml-api/26GlobalRequestElementsforListControl.html#ChoosingASubsetofRecords to control what data I would like back.

The first query is successful and when looking at the "matchingRecords" tells me:

startFrom = 1

maxNum = 500 (ceiling per request based on the doc)

totalRecords = 6471

For the next query, I modify the startFrom variable to 501 (since I already have the first group of 500) and post the request. The response status code is OK, however the returned value is just empty. Should I not be able to get all 6471 records by incrementing the startFrom element to collect 500 at a time?

Am I using the startFrom, maxNum, and totalRecords elements incorrectly?

Re: Choosing a Subset of Records

phillip.isley — Fri, 29 Jul 2016 19:32:07 GMT

After further investigation, my code is working as it should and there is simply an empty record. This shouldn't be something that happens, correct? I found this by testing various startFrom and setting maxNum to 1 so that the request would only ask for one entry. I found that startFrom = 520 returns a message response of OK, but the result is completely empty. This was causing any other request range that included 520 to "blow up" upon trying to deserialize it.

If a record exist, it should not be empty, correct? If this is something that is unavoidable, the only way I know to check for it is by requesting one record at a time and testing it's value. This would be fairly resource heavy since it's an individual POST for each record. Please tell me this isn't the only way to fix this.

Re: Choosing a Subset of Records

ryanhunt — Fri, 05 Aug 2016 17:11:44 GMT

What API Call are you using and what's your WebEx site? I will see if I can find out whats wrong with that record and blowing it up.

Re: Choosing a Subset of Records

phillip.isley — Fri, 05 Aug 2016 19:54:20 GMT

<?xml version="1.0" encoding="utf-8"?>
<serv:message
  xmlns:history="http://www.webex.com/schemas/2002/06/service/history"
  xmlns:com="http://www.webex.com/schemas/2002/06/common"
  xmlns:att="http://www.webex.com/schemas/2002/06/service/attendee"
  xmlns:meet="http://www.webex.com/schemas/2002/06/service/meeting"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:use="http://www.webex.com/schemas/2002/06/service/user"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:ep="http://www.webex.com/schemas/2002/06/service/ep"
  xmlns:serv="http://www.webex.com/schemas/2002/06/service">
  <serv:header>
       <serv:securityContext>
            <com:webExID>sas.admin</com:webExID>
            <com:password>XXXXX</com:password>
            <com:siteID>852782</com:siteID>
            <com:partnerID>guXv7Mx9mVjgvcsJzdh4_g</com:partnerID>
       </serv:securityContext>
  </serv:header>
  <serv:body>
       <serv:bodyContent xsi:type="ep:lstRecording">
            <ep:listControl>
                 <ep:startFrom>544</ep:startFrom>
                 <ep:maximumNum>1</ep:maximumNum>
            </ep:listControl>
            <ep:createTimeScope>
                 <ep:createTimeStart>01/01/2010 00:00:01</ep:createTimeStart>
                 <ep:createTimeEnd>02/05/2016 14:18:04</ep:createTimeEnd>
            </ep:createTimeScope>
            <ep:returnSessionDetails>true</ep:returnSessionDetails>
       </serv:bodyContent>
  </serv:body>
</serv:message>

https://sas.webex.com/

Currently, it's record 544 with the date range found in the XML. It is moving though since this is a production site and new records are being added regularly. I know my code is working correctly as a whole since I pulled 35k records from our old WebEx site successfully. This is a simply a matter of, why is this record empty instead of just not existing at all?

Re: Choosing a Subset of Records

ryanhunt — Tue, 09 Aug 2016 23:37:23 GMT

After speaking with our engineering team, it looks like one of your users: Antonio Gianni has a recording with an invalid XML character in the title, can you reach out to this user and get them to fix this recording name? It appears the problem is with a character between 'IT Enterprise Solutions' and 'Scheduling'

Re: Choosing a Subset of Records

phillip.isley — Wed, 10 Aug 2016 02:14:05 GMT

Thanks for the explanation Ryan. I will check with my user and test after a change. Do you have a list of invalid characters? Also, do you know why there is nothing in place to prevent users from entering an invalid character when saving a recording? This creates a big problem for the XML API since it returns an empty/null response. Would it be possible to open a bug report for this problem?

Re: Choosing a Subset of Records

ryanhunt — Wed, 10 Aug 2016 20:45:55 GMT

The problematic character is likely an em dash, or long hyphen.. The likely fix is to escape it into an XML Unicode mdash, I am still working w/engineering on a solution..

As a work around in the meantime, fixing this record manually should address the immediate issue.

Re: Choosing a Subset of Records

phillip.isley — Fri, 12 Aug 2016 15:02:17 GMT

Thanks again Ryan for all the help. I'm not sure what the actual character was, but I had my user modify the file name and it is no longer causing a problem. If possible, please keep me informed of when a final solution is implemented.

Re: Choosing a Subset of Records

phillip.isley — Tue, 18 Oct 2016 14:21:55 GMT

Hey Ryan,

Were there ever any changes made to address the issue of invalid XML characters used in Meeting/Recording names? I am running into this problem again and looking for a final solution. I haven't found a way to handle it from my side as the WebEx API doesn't return any data if an invalid character is present. I believe this is something that will have to be addressed from the WebEx side. The most likely solution is to escape all invalid XML characters before sending the response.

If I am missing something that I can do from my side, please let me know. I have had to revert back to processing the API requests in batches of 1 instead of 500 to ensure I don't exclude large chunks of data. As you can imagine, this greatly increases the time it takes for the full job to run. Please let me know if there is anything I can do to help be it testing, providing code, etc.

Thanks,

Phillip