cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3283
Views
3
Helpful
2
Replies

Find MediaSense Archive

Bruno Rangel
Spotlight
Spotlight

Hi All

My customer have a Mediasense 10.5 everything are working.

We configure the MediaSense Archive to customer´s SFTP and connection is fine and archiving as well.


But the customer need to find a record that is not present on MS Search&Play, so, we need find out on files that stored on SFTP, there are a lot file we are unable to understand how we can find this one...


Customer need find by Extension number and DateTime  that the record was done, the files are named as 20a149623655741.json and 20a149623655741.mp4.

How we can find/search specific call???

Cheers
Bruno Rangel
Please remember to rate helpful responses using the star bellow and identify helpful or correct answers
1 Accepted Solution

Accepted Solutions

dadawes
Level 1
Level 1

Bruno: good question, this should be pretty straightforward. I'm assuming you're using a Linux server as the sftp target to store the archive files, or if not Linux then you at least have a Linux compatible grep for the OS you are running.

Background

All of the archiving is done under the root directory configured in the MediaSense UI.

Starting from that directory, MediaSense creates a directory with the host name or IP address of the MediaSense node that is archiving records.

Under the hostname directory a directory with the archive date is created, and all the assorted references (*.json for metadata and *.mp4 for the actual media) are placed into the day's directory. On the following day another directory is created and used, and so on.

How you find a recording depends on whether you know which node in the cluster archived the record.


If you know which host had the recording

You can change to the host's directory. As an example, say I have set my archive root directory to /sftp.tmp and I know that host 10.194.118.227 archived the recording, I can ssh to my ftp/sftp server and run "cd /sftp.tmp/10.194.118.227 to get to the the "per day" directories. If I run ls (assuming Linux, or dir for WIndows but it will look a bit different and use \ rather than /) I see:

[root@ccbu-sj-vm228 10.194.118.227]# ls

20140927  20140928  20140929  20140930  20141002  20141104  20141107  20141110

This is an sftp server used for testing archive functions, so it only has entries for a few days.

If I'm looking (for example) for a recording from September 30 then I change into the 20140930 directory. From there an ls (or dir...) will show the assorted .json & .mp4 files archived for that day:

[root@ccbu-sj-vm228 20140930]# ls

637148c75b25801.json  639148c75dc7d91.json  63b148c7dd58c71.json

637148c75b25801.mp4   639148c75dc7d91.mp4   63b148c7dd58c71.mp4

638148c75cda181.json  63a148c7db1adf1.json

638148c75cda181.mp4   63a148c7db1adf1.mp4

You may see a fairly large pile of files, well into the thousands or even tens of thousands if your recording & archiving volume is high enough.


Searching by extension

We store the extension information in the deviceRef field, which looks like this:

"deviceRef": "70184089881739265"

To search for that specific extension I can use grep like this:

[root@ccbu-sj-vm228 20140930]# grep  "\"deviceRef\": \"70184089881739265\"" * -l

637148c75b25801.json

638148c75cda181.json

639148c75dc7d91.json

Replace the 70184089881739265 with the actual extension you are searching for.

Be careful to include the spaces and escape the embedded " characters properly.

In this case there are three archived recordings that include this extension.

Searching by time and date

The specific metadata that matches the time and date can be found if you know the exact date and time down to the second - you can simply grep for it. As an example, say I know the recording I want started at 30 Sep 2014 at 16:21:25 GMT. Using the conveient Epoch Converter - Unix Timestamp Converter web site I can convert that date and time to the number of milliseconds by entering 9 30 2014 for Month, Day and Yr, then enter 16 21 25 for Hr (using a 24 hour clock), Min and Sec, respectively. Hitting the convert button and reading out the time stamp in milliseconds I get 1412094085.

Note that this is in GMT time zone, the epoch converter can convert times and dates in other time zones if needed.

Next use grep with the converted epoch value as follows:

[root@ccbu-sj-vm228 20140930]# grep 1412094085 *.json -l

639148c75dc7d91.json

I now have the only file that had a start time and date within that second: 639148c75dc7d91.json. The matching 639148c75dc7d91.mp4 has the media for this recording. You may end up with more than one recording if the system is busy, in that case you'll have to open the .json files to figure out which one you want.

Searching by extension AND time and date

If the approaches above return too much data - which can happen if you're REALLY busy, you can search for both the time and date and extension by chaining grep as follows. Since the data spans multiple lines, you need to get the first grep to output the whole file, not just the match line, so that the second grep can then find the additional match if it is there. The two greps combined look like this:

grep  "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 | grep 1412094085 >match.txt

When the command completes, look at the match.txt file's contents. Each match will show the file name then a long stretch of JSON details, you just want to note the file names. My first match looks like this:

637148c75b25801.json:{  "callControllerIP": "10.194.3.53",  "callControllerType": "Cisco-SIPGateway",  "ccid": "1785577856-0000065536-0000060704-2541142538",  "sessionDuration": 44220,  "sessionId": "637148c75b25801",  "sessionStartDate": 1412093912699,  "sessionState": "CLOSED_NORMAL",  "tracks": [    {      "codec": "H264",      "participants": [        {          "deviceRef": "70184089881739265",          "isConference": false,          "participantDuration":

(etc. for many lines)

Note the bolded stuff at the beginning: 637148c75b25801.json this is the file that matched both greps. You can search for successive .json appearances to see if more than one file matched.

If you DON'T know which host had the recording

Unfortunately in this case you will need to check each of the possible hosts subdirectories using the techniques shown above. You could always try doing grep recursively from above the "per host" directories and look across all of them at once, but that may be too slow depending on how many files you have and how capable your sftp server is.


To search all of them at once using the combined double grep, you want to start in the root directory you have configured via MediaSense (/sftp.tmp in my example) and run the command as above with an additional -r flag:

grep  "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 -r | grep 1412094085 >match.txt


You can use any of the grep sequences above from the root dir with the added -r to search across all hosts and days, but this may be time consuming. You can always bail out of a long search by hitting <Ctrl>c if needed.


Hope that helps, let me know if you have any questions.

View solution in original post

2 Replies 2

dadawes
Level 1
Level 1

Bruno: good question, this should be pretty straightforward. I'm assuming you're using a Linux server as the sftp target to store the archive files, or if not Linux then you at least have a Linux compatible grep for the OS you are running.

Background

All of the archiving is done under the root directory configured in the MediaSense UI.

Starting from that directory, MediaSense creates a directory with the host name or IP address of the MediaSense node that is archiving records.

Under the hostname directory a directory with the archive date is created, and all the assorted references (*.json for metadata and *.mp4 for the actual media) are placed into the day's directory. On the following day another directory is created and used, and so on.

How you find a recording depends on whether you know which node in the cluster archived the record.


If you know which host had the recording

You can change to the host's directory. As an example, say I have set my archive root directory to /sftp.tmp and I know that host 10.194.118.227 archived the recording, I can ssh to my ftp/sftp server and run "cd /sftp.tmp/10.194.118.227 to get to the the "per day" directories. If I run ls (assuming Linux, or dir for WIndows but it will look a bit different and use \ rather than /) I see:

[root@ccbu-sj-vm228 10.194.118.227]# ls

20140927  20140928  20140929  20140930  20141002  20141104  20141107  20141110

This is an sftp server used for testing archive functions, so it only has entries for a few days.

If I'm looking (for example) for a recording from September 30 then I change into the 20140930 directory. From there an ls (or dir...) will show the assorted .json & .mp4 files archived for that day:

[root@ccbu-sj-vm228 20140930]# ls

637148c75b25801.json  639148c75dc7d91.json  63b148c7dd58c71.json

637148c75b25801.mp4   639148c75dc7d91.mp4   63b148c7dd58c71.mp4

638148c75cda181.json  63a148c7db1adf1.json

638148c75cda181.mp4   63a148c7db1adf1.mp4

You may see a fairly large pile of files, well into the thousands or even tens of thousands if your recording & archiving volume is high enough.


Searching by extension

We store the extension information in the deviceRef field, which looks like this:

"deviceRef": "70184089881739265"

To search for that specific extension I can use grep like this:

[root@ccbu-sj-vm228 20140930]# grep  "\"deviceRef\": \"70184089881739265\"" * -l

637148c75b25801.json

638148c75cda181.json

639148c75dc7d91.json

Replace the 70184089881739265 with the actual extension you are searching for.

Be careful to include the spaces and escape the embedded " characters properly.

In this case there are three archived recordings that include this extension.

Searching by time and date

The specific metadata that matches the time and date can be found if you know the exact date and time down to the second - you can simply grep for it. As an example, say I know the recording I want started at 30 Sep 2014 at 16:21:25 GMT. Using the conveient Epoch Converter - Unix Timestamp Converter web site I can convert that date and time to the number of milliseconds by entering 9 30 2014 for Month, Day and Yr, then enter 16 21 25 for Hr (using a 24 hour clock), Min and Sec, respectively. Hitting the convert button and reading out the time stamp in milliseconds I get 1412094085.

Note that this is in GMT time zone, the epoch converter can convert times and dates in other time zones if needed.

Next use grep with the converted epoch value as follows:

[root@ccbu-sj-vm228 20140930]# grep 1412094085 *.json -l

639148c75dc7d91.json

I now have the only file that had a start time and date within that second: 639148c75dc7d91.json. The matching 639148c75dc7d91.mp4 has the media for this recording. You may end up with more than one recording if the system is busy, in that case you'll have to open the .json files to figure out which one you want.

Searching by extension AND time and date

If the approaches above return too much data - which can happen if you're REALLY busy, you can search for both the time and date and extension by chaining grep as follows. Since the data spans multiple lines, you need to get the first grep to output the whole file, not just the match line, so that the second grep can then find the additional match if it is there. The two greps combined look like this:

grep  "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 | grep 1412094085 >match.txt

When the command completes, look at the match.txt file's contents. Each match will show the file name then a long stretch of JSON details, you just want to note the file names. My first match looks like this:

637148c75b25801.json:{  "callControllerIP": "10.194.3.53",  "callControllerType": "Cisco-SIPGateway",  "ccid": "1785577856-0000065536-0000060704-2541142538",  "sessionDuration": 44220,  "sessionId": "637148c75b25801",  "sessionStartDate": 1412093912699,  "sessionState": "CLOSED_NORMAL",  "tracks": [    {      "codec": "H264",      "participants": [        {          "deviceRef": "70184089881739265",          "isConference": false,          "participantDuration":

(etc. for many lines)

Note the bolded stuff at the beginning: 637148c75b25801.json this is the file that matched both greps. You can search for successive .json appearances to see if more than one file matched.

If you DON'T know which host had the recording

Unfortunately in this case you will need to check each of the possible hosts subdirectories using the techniques shown above. You could always try doing grep recursively from above the "per host" directories and look across all of them at once, but that may be too slow depending on how many files you have and how capable your sftp server is.


To search all of them at once using the combined double grep, you want to start in the root directory you have configured via MediaSense (/sftp.tmp in my example) and run the command as above with an additional -r flag:

grep  "\"deviceRef\": \"70184089881739265\"".* * -B 1000 -A 1000 -r | grep 1412094085 >match.txt


You can use any of the grep sequences above from the root dir with the added -r to search across all hosts and days, but this may be time consuming. You can always bail out of a long search by hitting <Ctrl>c if needed.


Hope that helps, let me know if you have any questions.

Thanks for your explanation, it was I'm looking for...

Cheers
Bruno Rangel
Please remember to rate helpful responses using the star bellow and identify helpful or correct answers