11-12-2014 04:10 PM
Hi All
My customer have a Mediasense 10.5 everything are working.
We configure the MediaSense Archive to customer´s SFTP and connection is fine and archiving as well.
But the customer need to find a record that is not present on MS Search&Play, so, we need find out on files that stored on SFTP, there are a lot file we are unable to understand how we can find this one...
Customer need find by Extension number and DateTime that the record was done, the files are named as 20a149623655741.json and 20a149623655741.mp4.
How we can find/search specific call???
Solved! Go to Solution.
11-12-2014 05:25 PM
Bruno: good question, this should be pretty straightforward. I'm assuming you're using a Linux server as the sftp target to store the archive files, or if not Linux then you at least have a Linux compatible grep for the OS you are running.
Background
All of the archiving is done under the root directory configured in the MediaSense UI.
Starting from that directory, MediaSense creates a directory with the host name or IP address of the MediaSense node that is archiving records.
Under the hostname directory a directory with the archive date is created, and all the assorted references (*.json for metadata and *.mp4 for the actual media) are placed into the day's directory. On the following day another directory is created and used, and so on.
How you find a recording depends on whether you know which node in the cluster archived the record.
If you know which host had the recording
You can change to the host's directory. As an example, say I have set my archive root directory to /sftp.tmp and I know that host 10.194.118.227 archived the recording, I can ssh to my ftp/sftp server and run "cd /sftp.tmp/10.194.118.227 to get to the the "per day" directories. If I run ls (assuming Linux, or dir for WIndows but it will look a bit different and use \ rather than /) I see:
[root@ccbu-sj-vm228 10.194.118.227]# ls
20140927 20140928 20140929 20140930 20141002 20141104 20141107 20141110
This is an sftp server used for testing archive functions, so it only has entries for a few days.
If I'm looking (for example) for a recording from September 30 then I change into the 20140930 directory. From there an ls (or dir...) will show the assorted .json & .mp4 files archived for that day:
[root@ccbu-sj-vm228 20140930]# ls
637148c75b25801.json 639148c75dc7d91.json 63b148c7dd58c71.json
637148c75b25801.mp4 639148c75dc7d91.mp4 63b148c7dd58c71.mp4
638148c75cda181.json 63a148c7db1adf1.json
638148c75cda181.mp4 63a148c7db1adf1.mp4
You may see a fairly large pile of files, well into the thousands or even tens of thousands if your recording & archiving volume is high enough.
Searching by extension
We store the extension information in the deviceRef field, which looks like this:
"deviceRef": "70184089881739265"
To search for that specific extension I can use grep like this:
[root@ccbu-sj-vm228 20140930]# grep "\"deviceRef\": \"70184089881739265\"" * -l
637148c75b25801.json
638148c75cda181.json
639148c75dc7d91.json
Replace the 70184089881739265 with the actual extension you are searching for.
Be careful to include the spaces and escape the embedded " characters properly.
In this case there are three archived recordings that include this extension.
Searching by time and date
The specific metadata that matches the time and date can be found if you know the exact date and time down to the second - you can simply grep for it. As an example, say I know the recording I want started at 30 Sep 2014 at 16:21:25 GMT. Using the conveient Epoch Converter - Unix Timestamp Converter web site I can convert that date and time to the number of milliseconds by entering 9 30 2014 for Month, Day and Yr, then enter 16 21 25 for Hr (using a 24 hour clock), Min and Sec, respectively. Hitting the convert button and reading out the time stamp in milliseconds I get 1412094085.
Note that this is in GMT time zone, the epoch converter can convert times and dates in other time zones if needed.
Next use grep with the converted epoch value as follows:
[root@ccbu-sj-vm228 20140930]# grep 1412094085 *.json -l
639148c75dc7d91.json
I now have the only file that had a start time and date within that second: 639148c75dc7d91.json. The matching 639148c75dc7d91.mp4 has the media for this recording. You may end up with more than one recording if the system is busy, in that case you'll have to open the .json files to figure out which one you want.
Searching by extension AND time and date
If the approaches above return too much data - which can happen if you're REALLY busy, you can search for both the time and date and extension by chaining grep as follows. Since the data spans multiple lines, you need to get the first grep to output the whole file, not just the match line, so that the second grep can then find the additional match if it is there. The two greps combined look like this:
grep "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 | grep 1412094085 >match.txt
When the command completes, look at the match.txt file's contents. Each match will show the file name then a long stretch of JSON details, you just want to note the file names. My first match looks like this:
637148c75b25801.json:{ "callControllerIP": "10.194.3.53", "callControllerType": "Cisco-SIPGateway", "ccid": "1785577856-0000065536-0000060704-2541142538", "sessionDuration": 44220, "sessionId": "637148c75b25801", "sessionStartDate": 1412093912699, "sessionState": "CLOSED_NORMAL", "tracks": [ { "codec": "H264", "participants": [ { "deviceRef": "70184089881739265", "isConference": false, "participantDuration":
(etc. for many lines)
Note the bolded stuff at the beginning: 637148c75b25801.json this is the file that matched both greps. You can search for successive .json appearances to see if more than one file matched.
If you DON'T know which host had the recording
Unfortunately in this case you will need to check each of the possible hosts subdirectories using the techniques shown above. You could always try doing grep recursively from above the "per host" directories and look across all of them at once, but that may be too slow depending on how many files you have and how capable your sftp server is.
To search all of them at once using the combined double grep, you want to start in the root directory you have configured via MediaSense (/sftp.tmp in my example) and run the command as above with an additional -r flag:
grep "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 -r | grep 1412094085 >match.txt
You can use any of the grep sequences above from the root dir with the added -r to search across all hosts and days, but this may be time consuming. You can always bail out of a long search by hitting <Ctrl>c if needed.
Hope that helps, let me know if you have any questions.
11-12-2014 05:25 PM
Bruno: good question, this should be pretty straightforward. I'm assuming you're using a Linux server as the sftp target to store the archive files, or if not Linux then you at least have a Linux compatible grep for the OS you are running.
Background
All of the archiving is done under the root directory configured in the MediaSense UI.
Starting from that directory, MediaSense creates a directory with the host name or IP address of the MediaSense node that is archiving records.
Under the hostname directory a directory with the archive date is created, and all the assorted references (*.json for metadata and *.mp4 for the actual media) are placed into the day's directory. On the following day another directory is created and used, and so on.
How you find a recording depends on whether you know which node in the cluster archived the record.
If you know which host had the recording
You can change to the host's directory. As an example, say I have set my archive root directory to /sftp.tmp and I know that host 10.194.118.227 archived the recording, I can ssh to my ftp/sftp server and run "cd /sftp.tmp/10.194.118.227 to get to the the "per day" directories. If I run ls (assuming Linux, or dir for WIndows but it will look a bit different and use \ rather than /) I see:
[root@ccbu-sj-vm228 10.194.118.227]# ls
20140927 20140928 20140929 20140930 20141002 20141104 20141107 20141110
This is an sftp server used for testing archive functions, so it only has entries for a few days.
If I'm looking (for example) for a recording from September 30 then I change into the 20140930 directory. From there an ls (or dir...) will show the assorted .json & .mp4 files archived for that day:
[root@ccbu-sj-vm228 20140930]# ls
637148c75b25801.json 639148c75dc7d91.json 63b148c7dd58c71.json
637148c75b25801.mp4 639148c75dc7d91.mp4 63b148c7dd58c71.mp4
638148c75cda181.json 63a148c7db1adf1.json
638148c75cda181.mp4 63a148c7db1adf1.mp4
You may see a fairly large pile of files, well into the thousands or even tens of thousands if your recording & archiving volume is high enough.
Searching by extension
We store the extension information in the deviceRef field, which looks like this:
"deviceRef": "70184089881739265"
To search for that specific extension I can use grep like this:
[root@ccbu-sj-vm228 20140930]# grep "\"deviceRef\": \"70184089881739265\"" * -l
637148c75b25801.json
638148c75cda181.json
639148c75dc7d91.json
Replace the 70184089881739265 with the actual extension you are searching for.
Be careful to include the spaces and escape the embedded " characters properly.
In this case there are three archived recordings that include this extension.
Searching by time and date
The specific metadata that matches the time and date can be found if you know the exact date and time down to the second - you can simply grep for it. As an example, say I know the recording I want started at 30 Sep 2014 at 16:21:25 GMT. Using the conveient Epoch Converter - Unix Timestamp Converter web site I can convert that date and time to the number of milliseconds by entering 9 30 2014 for Month, Day and Yr, then enter 16 21 25 for Hr (using a 24 hour clock), Min and Sec, respectively. Hitting the convert button and reading out the time stamp in milliseconds I get 1412094085.
Note that this is in GMT time zone, the epoch converter can convert times and dates in other time zones if needed.
Next use grep with the converted epoch value as follows:
[root@ccbu-sj-vm228 20140930]# grep 1412094085 *.json -l
639148c75dc7d91.json
I now have the only file that had a start time and date within that second: 639148c75dc7d91.json. The matching 639148c75dc7d91.mp4 has the media for this recording. You may end up with more than one recording if the system is busy, in that case you'll have to open the .json files to figure out which one you want.
Searching by extension AND time and date
If the approaches above return too much data - which can happen if you're REALLY busy, you can search for both the time and date and extension by chaining grep as follows. Since the data spans multiple lines, you need to get the first grep to output the whole file, not just the match line, so that the second grep can then find the additional match if it is there. The two greps combined look like this:
grep "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 | grep 1412094085 >match.txt
When the command completes, look at the match.txt file's contents. Each match will show the file name then a long stretch of JSON details, you just want to note the file names. My first match looks like this:
637148c75b25801.json:{ "callControllerIP": "10.194.3.53", "callControllerType": "Cisco-SIPGateway", "ccid": "1785577856-0000065536-0000060704-2541142538", "sessionDuration": 44220, "sessionId": "637148c75b25801", "sessionStartDate": 1412093912699, "sessionState": "CLOSED_NORMAL", "tracks": [ { "codec": "H264", "participants": [ { "deviceRef": "70184089881739265", "isConference": false, "participantDuration":
(etc. for many lines)
Note the bolded stuff at the beginning: 637148c75b25801.json this is the file that matched both greps. You can search for successive .json appearances to see if more than one file matched.
If you DON'T know which host had the recording
Unfortunately in this case you will need to check each of the possible hosts subdirectories using the techniques shown above. You could always try doing grep recursively from above the "per host" directories and look across all of them at once, but that may be too slow depending on how many files you have and how capable your sftp server is.
To search all of them at once using the combined double grep, you want to start in the root directory you have configured via MediaSense (/sftp.tmp in my example) and run the command as above with an additional -r flag:
grep "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 -r | grep 1412094085 >match.txt
You can use any of the grep sequences above from the root dir with the added -r to search across all hosts and days, but this may be time consuming. You can always bail out of a long search by hitting <Ctrl>c if needed.
Hope that helps, let me know if you have any questions.
01-30-2015 04:28 AM
Thanks for your explanation, it was I'm looking for...
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide