UCSD and Git, Better together

mt_solutions · ‎06-13-2017

Hi all,

we are a small HK based partner and we build some interesting integrations for UCSD and GIT, which makes our lives easier on our day to day develovpment:

MT Solutions - UCSD - GIT Integration on Vimeo

Hope you like it!

Kral Two · ‎06-13-2017

That's great!

what should it take to have a webhook calling the UCSD pull workflow when the repo is updated? :-)

Orf Gelbrich · ‎06-13-2017

You would need to run a scheduled workflow on UCSD that checks Git every few seconds.

Kral Two · ‎06-14-2017

I was thinking about calling a UCSD workflow via northbound API, when the repository see an update (action/module to be done on the repository side, gitlab, gogs, etc ...)

Orf Gelbrich · ‎06-14-2017

That would work as well if your UCSD instance is addressable from the outside world.

Orf Gelbrich · ‎06-13-2017

Now let it be noted in UCSD 6.5 you can check in and out workflows to Git.

ekleditzpge · ‎08-03-2017

Im not sure this is the case. I just upgraded to 6.5 and there is no ability (that I could find anyway) to check flows in and out of git. You can pull custom task scripts out of git, but thats it. It also appears to only work with github. I was unable to get it to talk to our internal git server. I do have a ticket open on that though.

Pretty disappointing if thats the only git integration and Im really hoping that im just missing things.

Orf Gelbrich · ‎08-05-2017

Try this:

ekleditzpge · ‎08-07-2017

Hi Orf,

Thanks for the response. And thanks for all the great stuff you have put up here the last few years. I have used lots of it.

Does this integration only work with github servers? Our internal repos are all bitbucket.

Also you mentioned above that we would be able to check flows in and out. Is that the case? I cant fine that anywhere.

zaleski.krzysztof · ‎06-16-2017

Hello,

What you do here is a simple backup. You could have used FTP or anything else, as well. GIT is ment to be a code repository, it's not very suitable for binary files. What I would rather see (will come with 6.5, but still limited) is a possibility to update the source code of a custom task locally, push it to the git, and then UCSD should get the code and apply to the custom task. It can be done directly via the UCSD or maybe with Jenkins fetching the code, adding to UCSD and even running some tests with temporary data (pure devops style). So, you do not have to go thgough all GUI options of the custom task when you want to just update the javascript code. The main issue is that the worflow is a binary file, and you cannot upload remotely just the code.

Cheers,

Krzysztof

gene.williams · ‎04-24-2018

Hey Orf,

Do you have answers for Erich's questions (9.) ?

Orf Gelbrich · ‎04-24-2018

In here https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-director/custom-task-mgmt-guide/6-5/cisco-ucs-director-custom-task-getting-started-65/Cisco-UCS-Director-Custom-Task-Getting-Started-Guide-7-0_chapter_011.html?bookSearch=true

They only talk about github.

But I would turn on debug and connect and turn debug off and see what else you can find in the log.

gene.williams · ‎04-25-2018

The PDF in the link below says "or your own private enterprise Git repository" at item one under "Custom Tasks and Git Repositories" on page 20. Initially, nothing sticks out in the debug logs, but I'll keep plugging at it from different angles.

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-director/custom-task-mgmt-guide/6-5/cisco-ucs-director-custom-task-getting-started-65.pdf

Orf Gelbrich · ‎04-25-2018

Did you turn on debug logging in administration -> Support information -> debug logging

And the run a tail –f /opt/infra/inframgr/logfile.txt

gene.williams · ‎05-04-2018

I did and captured the catalina.log, CloupiaServiceClient.log, and the inframgr.log, debug mode started just before the connection test and stopped immediately after. Only the inframgr log had anything obviously related to the connection test - and no tell-tale cause for only the authentication part of the test failing. Below is entire inframgr.log during the debug capture. Again, the ping test and connection test parts succeed - only the Login is unsuccessful. I've repeatedly re-entered these credentials and triple checked them logging into the git repo via web browser.

Is it possible the connection test input field might not like certain special characters in a password?

2018-05-04 15:39:04,567 [ORP-1-34] INFO startRecording(LogRecorder.java:145) - Start recording logs!

2018-05-04 15:39:04,569 [ORP-1-34] INFO run(OperationRequestProcessor.java:138) - Sending response... requestId=22237

2018-05-04 15:39:04,569 [ORP-1-34] DEBUG sendReply(LANServerChannel.java:80) - sending reply ...

2018-05-04 15:39:04,916 [WorkFlowManager] DEBUG query(JDBCObjStore.java:195) - Query table --> limit -1 offset -1

2018-05-04 15:39:04,916 [WorkFlowManager] DEBUG query(JDBCObjStore.java:204) - Executing query (MySQL)

2018-05-04 15:39:04,916 [WorkFlowManager] DEBUG parseJDOQL(JDBCObjStore.java:816) - Incase of SQLInjection Excpetion below, query executed by POJO:class com.cloupia.model.cIM.WorkFlow

2018-05-04 15:39:04,916 [WorkFlowManager] DEBUG convertToJDBCQuery(QueryParser.java:106) - jdoQuery priority <= 1 && adminStatus == 1 && executionStatus <= 1 && futureStartTime <= 1525448344916

2018-05-04 15:39:04,916 [WorkFlowManager] DEBUG query(JDBCObjStore.java:229) - Executing query SELECT * FROM WORKFLOW_DETAILS A WHERE priority <= 1 AND adminStatus = 1 AND executionStatus <= 1 AND futureStartTime <= 1525448344916

2018-05-04 15:39:04,922 [WorkFlowManager] DEBUG query(JDBCObjStore.java:258) - Query result 0 rows, time =6 ms

2018-05-04 15:39:04,923 [WorkFlowManager] DEBUG startWorkFlowProcessing(WorkFlowManager.java:827) - Started processing Workflows, count: 0

2018-05-04 15:39:07,654 [ORR-InfraMgr:D941DC03FB] DEBUG getNextTransaction(LANServerChannel.java:68) - received transation for requestId 22238

2018-05-04 15:39:07,654 [ORR-InfraMgr:D941DC03FB] INFO run(OperationRequestReceiver.java:38) - Received operational request, added to queue.

2018-05-04 15:39:07,655 [ORP-1-46] INFO run(OperationRequestProcessor.java:77) - Executing request... requestId=22238; opName=validateForm

2018-05-04 15:39:07,655 [ORP-1-46] INFO validateForm(APIProvider.java:1177) - FORM VALIDATE SystemSettings.gitrepo_edit_form

2018-05-04 15:39:07,656 [ORP-1-46] INFO testURL(AddRepoForm.java:267) - Ping check to stash.ercot.com

2018-05-04 15:39:07,658 [ORP-1-46] INFO testURL(AddRepoForm.java:278) - Host stash.ercot.com is reachable.

2018-05-04 15:39:07,658 [ORP-1-46] INFO testURL(AddRepoForm.java:287) - Connection check...

2018-05-04 15:39:07,923 [WorkFlowManager] DEBUG query(JDBCObjStore.java:195) - Query table --> limit -1 offset -1

2018-05-04 15:39:07,923 [WorkFlowManager] DEBUG query(JDBCObjStore.java:204) - Executing query (MySQL)

2018-05-04 15:39:07,923 [WorkFlowManager] DEBUG parseJDOQL(JDBCObjStore.java:816) - Incase of SQLInjection Excpetion below, query executed by POJO:class com.cloupia.model.cIM.WorkFlow

2018-05-04 15:39:07,923 [WorkFlowManager] DEBUG convertToJDBCQuery(QueryParser.java:106) - jdoQuery priority <= 0 && adminStatus == 1 && executionStatus <= 1 && futureStartTime <= 1525448347923

2018-05-04 15:39:07,923 [WorkFlowManager] DEBUG query(JDBCObjStore.java:229) - Executing query SELECT * FROM WORKFLOW_DETAILS A WHERE priority <= 0 AND adminStatus = 1 AND executionStatus <= 1 AND futureStartTime <= 1525448347923

2018-05-04 15:39:07,929 [WorkFlowManager] DEBUG query(JDBCObjStore.java:258) - Query result 0 rows, time =6 ms

2018-05-04 15:39:07,930 [WorkFlowManager] DEBUG startWorkFlowProcessing(WorkFlowManager.java:827) - Started processing Workflows, count: 0

2018-05-04 15:39:08,340 [ORP-1-46] INFO testURL(AddRepoForm.java:296) - Connected successfully to: https://stash.ercot.com/projects/UCSD

2018-05-04 15:39:08,342 [ORP-1-46] INFO validateLogin(AddRepoForm.java:165) - Test login to stash.ercot.com/projects/UCSD

2018-05-04 15:39:08,754 [ORP-1-46] INFO run(OperationRequestProcessor.java:138) - Sending response... requestId=22238

2018-05-04 15:39:08,754 [ORP-1-46] DEBUG sendReply(LANServerChannel.java:80) - sending reply ...

2018-05-04 15:39:10,930 [WorkFlowManager] DEBUG query(JDBCObjStore.java:195) - Query table --> limit -1 offset -1

2018-05-04 15:39:10,930 [WorkFlowManager] DEBUG query(JDBCObjStore.java:204) - Executing query (MySQL)

2018-05-04 15:39:10,930 [WorkFlowManager] DEBUG parseJDOQL(JDBCObjStore.java:816) - Incase of SQLInjection Excpetion below, query executed by POJO:class com.cloupia.model.cIM.WorkFlow

2018-05-04 15:39:10,930 [WorkFlowManager] DEBUG convertToJDBCQuery(QueryParser.java:106) - jdoQuery priority <= 1 && adminStatus == 1 && executionStatus <= 1 && futureStartTime <= 1525448350930

2018-05-04 15:39:10,930 [WorkFlowManager] DEBUG query(JDBCObjStore.java:229) - Executing query SELECT * FROM WORKFLOW_DETAILS A WHERE priority <= 1 AND adminStatus = 1 AND executionStatus <= 1 AND futureStartTime <= 1525448350930

2018-05-04 15:39:10,936 [WorkFlowManager] DEBUG query(JDBCObjStore.java:258) - Query result 0 rows, time =6 ms

2018-05-04 15:39:10,937 [WorkFlowManager] DEBUG startWorkFlowProcessing(WorkFlowManager.java:827) - Started processing Workflows, count: 0