Need to automate the jenkins CICD work flow for Azure dataBricks?
We need to understand how to DevOps process works to get Databricks processes automated.
What is our source control process, jenkins build process, and job triggering process from control-m if a developer is coding a spark job in Scala, Python or R.
DoD: Example projects / jenkins pipelines for a Scala, Python and R Spark project with the job triggered via REST / DataBricks API. Should be a total of 3 projects.