Tidal: Contents of Tidal variable exposed in Job Activities panel, any way of shutting this off?

[Dave.Carnahan]

(Good or bad)

In TESv6.2.1, I'm using several Tidal variables to contain some credentials to a few Tidal jobs.

The Tidal variables are not declared as 'public', and I'm not permitting Tidal users access into the Variables area of Tidal (to allow the users see these objects and their values).

And in the Job Activity panel, the variables aren't exposed until after job execution. (I would expect they never be exposed, but I have what I have)

After execution, those values are clearly exposed to the Tidal group members, specially in the parameters section of the job within the Job Activities section

Any suggestions on methods to either mask the variable values in the Job Activities pane after the job has executed or not permit exposure altogether? 

Thanks in advance-

[Tyler Theobald]

My method when I have had to pass credentials in TES jobs, but not want anyone else to see.

- Pass them in a batch file, which standard users don't have READ permissions to.

- In the batch file, on the line that as the username/password, prefix with '@' and it won't be echo'd to standard output.

Is that an option for you? 

[Dave.Carnahan]

Might be a start, (sort of...IDK)

If I keep credentials in a file on a Tidal agent server where a Tidal agent can read them, that starts to introduce managing credentials outside of the non-centralized Tidal environment.

[Thinking out loud]

I'm wondering if Tidal can allow placing Tidal variables - inside Tidal variables, for the purpose of eliminating 'variable-value' exposure in the JobRun / Override Tab?

--

In the big picture, I'm starting to open up Tidal to the developers supporting specific projects where they have batch jobs running, by having them use the Tidal web (or Java) client.

These developers are my day-to-day customer, and to increase Tidal value in my shop (and demonstrate transparency) I need to get the developers involved with their jobs (to one level or another).

But the deal is those credentials are something they shouldn't have access to because they're production values and not dev values.

That's the catch in all of this.

Thanks for the thoughts, keep em' coming!

[Dave.Carnahan]

Also, in my environment, I have 4 different Tidal agent servers, so I'd have to manage that one file on four different servers, where their environment isn't completely vanilla*.

*-Three Windows servers have a share on the C: drive for Tidal (two Windows 2008R2 Standard Edition, one 2008R2 Enterprise), the fourth has a T: drive (Windows 2012 Std edition).

Ugh... I have to (basically) write shims to compensate for this difference, but at least my network manager still keeps giving my Tidal projects newer boxes to run the Tidal jobs on and keeps us working <thumbs up>!

[Tyler Theobald]

More thoughts for your thinking!

- the "credential" files don't have to be replicated on 4 different agent servers.  Your process/job can all be referencing the same file via UNC path to a file server (for instance).

- Also, sounds like you are getting where we've been for years.  I keep the TES environments working, but the developer teams are in control of their own jobs.

- We do our best to never have to pass credentials, but use "trusted authentication" to network resources (files, SQL) using the runtime user the TES job is already running under. None of my developers know the passwords of the TES runtime accounts.  

- Also, I feel your pain on having different TES agent server configurations.  Though we have some variation with agent servers owned by different teams, the main agent servers under my stewardship are all identical servers (Windows Server 2012), with jobs load balanced across the balanced agent list. 

[Dave.Carnahan]

Yep, for years we centralized the Tidal activities and development units would release Tidal jobs to us to maintain and respond to.

We'd write formatted console output from the Tidal jobs to essentially email back to a Tidal workflow / distribution list and these guys never saw the layer I'd built for them - except that I'd simply run their jobs they handed off to my area.

Here's another challenge - with credentials - 

With SQL, the workflows come at it with SQL and AD creds.

For Directory Service (file shares), we'd setup the target share to permit our 'Tidal' ID  access, which is central to all the jobs. I've since turned that feature off in Tidal v6.2.1 to enable different AD ids for impersonation. We'll see how this goes later, right now, we're running everything still the same way, but I have the extra step to ensure my one ID is the one running all the jobs.

But other software, like WS_FTP Pro, we've used since '06 because of its scriptability and have differing credentials for the different workflow/projects.

This is where it'll hurt the most.

Our organization relies on working with our outside partnerships and those credentials make us a guests into our partners systems, not ours... 

I'd halfway think about using Tidal's FTP entities, except for the 1-shot designed action. 

Instead, check this out with WS_FTP command set -

http://ipswitchft.force.com/kb/#!/article/How-to-delete-move-rename-the-source-file-after-transfer

By writing a dynamic script, I can determine and write a temp script to fit the need of the actual FTP requirement, invoke the batch WS_FTP with this temp script, then throw away the temp script commands afterwards and no passwords are stored in the files laying around, creds are sent down to the Tidal job parameters.*

*-It's just that the values show up in the Tidal source, after they've executed.

Good for letting me let other DevOps into Tidal jobs to see this (#NOT, #SNOWDEN...).

There may be something in PowerShell where I can store these credentials on the server with .NetFramework, (within PowerShell see help in the cmdlet 'Get-Credential') but to combine this with Tidal is going to be a challenge that I'd like to use the time for something else for.

Ok, I'm off my soapbox. Thx ... 

[Dave.Carnahan]

Solution: ? 

If Cisco built into its Tidal software,

A switch (on each job definition) that says to omit the exposure of variables values from the console viewed content... that would work