Abstract: Systems, computer program products, and methods for interactive automated code generation and modification for data processing are provided. The method includes causing a rendering of a flow designer interface including a flow portion and a plugin portion. The plugin portion includes engageable plugin icon(s) corresponding to one or more plugins. The method also includes receiving a first plugin input based on engagement of a first plugin icon. The first plugin input is a selection of a first plugin. The method further includes causing a rendering of a representation of the first plugin on the flow portion. The method further includes receiving a second plugin input based on engagement of a second plugin icon. The method also includes causing a rendering of a representation of the second plugin on the flow portion. The method also includes generating a flow operation based on the flow portion of the flow designer interface.

Abstract: Systems, computer program products, and methods for determining and maintaining data quality in data processing is provided. The method includes determining one or more plugins used in a flow execution. The flow execution performs at least one transformation on one or more data sets. The method also includes determining one or more automatic data quality rules based on at least one of the one or more plugins using in the flow execution. The method further includes determining one or more custom data quality rules based on a custom data quality rule input received from an end-point device. The custom data quality rule includes one or more data statistics to be monitored during the flow execution. The method still further includes causing execution of each of the automatic data quality rules and the one or more custom data quality rules on data in the flow execution.

Abstract: An infrastructure system facilitates trace-based surveys to query users their experiences with pull request workflows. For example, the infrastructure system may receive from a client device associated with a user a pull request for making one or more changes to a programming code repository. The infrastructure system may generate a trace based on a pull request workflow including operations for making the change(s). The trace includes spans, each of which correspond to one or more operations in the pull request workflow. The infrastructure system generates log files, each log file includes data logged during the one or more operations of a span. The log files may be stored as a dataset of the trace. The infrastructure system may transmit a survey to the client device based on the dataset. The infrastructure system may also modify the pull request workflow based on the user's response to the survey.

Abstract: Failures can occur when a software process is running, or when the software process is undergoing tests. A remote debugging session can be created to run and inspect the software process, so that a developer can fix the bug that caused a failure. Remote debugging sessions can consume valuable resources on cluster infrastructure. Debugging the failures can take up a significant amount of developer time. To alleviate some of these problems, a remote debugging session can be created on-demand at a later time when a developer is ready to start the remote debugging session. The deferred creation of the remote debugging session can be triggered by activating a uniform resource identifier that has information for creating the remote debugging session encoded therein. The resulting implementation makes it easy for a developer to create a remote debugging session, and resources can be allocated on an as-needed basis.

Abstract: The present disclosure may use reverse forwarded socket connections and rules to manage program code that is being developed. Each developer responsible for developing program code may be provided unique login information that they may use to access data associated with a development environment. Validations of a user/developer and/or computer identity may be performed before a user is allowed to access and update sets of program code. Reverse forwarded socket connections may be associated with endpoints at a remote computer. Once a particular connection is established and a validation performed, identifiers may be used to direct communications to processes, sub-processes, virtual machines, or specific computing resources that are associated with the development environment. By using reverse forwarded socket connections, computers that run different types of operating system software may be used to develop program code without having to install specialized software on those computers.