Dataiku offers a robust platform for data processing and machine learning, equipped with a plethora of functionalities that make it simple for users to manage, examine, and visualize data. One of its most useful features, “App-as-Recipe,” enables users to bundle multiple components as a single entity. This functionality promotes reusability, uniformity, and control in data projects, streamlining data input standardization and facilitating access to specific procedures in various workflows.
In this article, we’ll delve into the details of Dataiku’s app-as-recipe feature, demonstrating its usage, presenting examples of potential applications, and sharing best practices for effective implementation. We will discuss how app-as-recipe simplifies the creation of reusable components and enhances workflow maintenance and modification over time. We’ll also cover the setup process for an app-as-recipe, parameter and input configuration, and its integration into other flows. Additionally, we’ll explore best practices for optimizing recipes for reusability, app documentation to improve comprehension, and determining when to use app-as-recipe versus developing a plugin.
By the end of this article, you’ll better understand how app-as-recipe can optimize your workflows and enhance your data projects’ efficiency on Dataiku. Whether you’re an experienced Dataiku user or just beginning, the “App-as-Recipe” feature can simplify your data workflows and help you derive more value from your data projects.
What is an App-as-Recipe in Dataiku?
Dataiku’s App-as-Recipe functionality is a potent instrument that allows data professionals to package multiple components as a singular unit. This feature lets you create a template or recipe that can be reused across various workflows or projects, offering several key advantages, such as reusability, uniformity, and control.
Reusability is a core benefit of App-as-Recipe. Creating a single recipe lets you reuse the same steps in numerous workflows, saving time and effort. For instance, if you have a series of steps that you must repeatedly perform across different workflows, you can create an App-as-Recipe that can effortlessly integrate into each workflow.
Consistency is another critical advantage of App-as-Recipe. Creating a standardized set of steps that can be reused across multiple workflows ensures that your data processing is consistent and dependable.
Governability is the third main advantage of App-as-Recipe. Bundling multiple components as a single entity allows you to create a governed SSOT (single source of transformation) that multiple people across different workflows can use. This guarantees that everyone uses the same steps and that any recipe alterations or updates are made centrally.
You can also restrict an App-as-Recipe so that only specific individuals can edit it. This ensures that a select group of people can make changes to the recipe while broader usage of the App-as-Recipe is available to the group.
Examples & Use Cases for the App-as-Recipe
Are you curious about how to apply App-as-Recipe and the scenarios in which it can be employed? App-as-Recipe is easy to use and adaptable, offering various enhancements to your data workflows. Consider the following applications of App-as-Recipe:
Standardizing Data Inputs: A typical scenario for App-as-Recipe is ensuring uniform data inputs. Suppose your team is involved in a machine learning project requiring data preprocessing before modeling. In this case, App-as-Recipe can be created to standardize data inputs throughout the project, ensuring consistent preprocessing and increasing the accuracy of the outcomes.
Automating Workflows: App-as-Recipe can be used to automate workflows. For instance, if a complex workflow involves multiple stages, an App-as-Recipe can be designed to automate the entire procedure. This approach saves time, minimizes effort, and decreases the likelihood of errors or inconsistencies in the workflow.
Sharing Component: Another advantage of App-as-Recipe is the ability to share components across various workflows. A specific component used in different workflows can be packaged as an App-as-Recipe and shared with team members. This strategy maintains consistency and eliminates the need to recreate identical components in separate workflows. For instance, imagine your data science team is tackling various marketing analytics projects, with each project necessitating identical data preprocessing procedures like data cleaning, addressing missing values, and encoding categorical data. Rather than replicating these steps in each project’s workflow, you can develop an App-as-Recipe that encompasses these shared preprocessing tasks.
How to Set Up & Use Dataiku's App-as-Recipe
1. Start by selecting a project: Choose an existing project with a workflow you want to turn into an app or create a new project containing the desired workflow.
2. Convert to App-as-Recipe: In the Dataiku project, go to the More Options (“…”) -> Application Designer, and select “Convert into application-as-recipe. Provide a title and description for your app and choose a folder for it.
3. Design the App: In the app’s interface, build the necessary components, including input datasets, processing stages, and output datasets. Ensure they are modular and easily integrated into other workflows.
4. Create the Scenario: Save the App. Go to the Jobs menu, and create a new scenario. Add a Build/Train step, and select output data as the dataset to build. Save the scenario, and return to the Application Designer. Set “Run recipe” as the scenario for the app.
5. Set Parameters and customize: In the app settings, define parameters and inputs that users can modify when using the app as a recipe. This allows users to adapt the app to their needs while maintaining its core functionality. In the Application Designer, click “Use custom UI” to add instructions for the app user. Write custom UI HTML code outlining the expected schema of the input dataset.
6. Save and Publish the App: Once the app is finalized, save and publish it. This action makes the app accessible as a recipe in other Dataiku projects.
7. Utilize the App-as-Recipe: To use the app as a recipe in a different Dataiku project, create a new recipe and choose the app from the list of available recipes. Adjust the parameters and inputs, and connect the input and output datasets. Ensure the input data has the same schema as the input data in the app. The app will now serve as a reusable component in your project’s workflow.
By following these steps, you can implement and use an app-as-recipe to optimize your data workflows and enhance your project’s efficiency.
Best Practices for Operating App-as-Recipe
To fully utilize the App-as-Recipe feature, it is crucial to adhere to some best practices. Here are several suggestions for effectively using App-as-Recipe:
Optimize your recipes for reusability: When creating a recipe, aim for reusability across various workflows and users. Adopt a modular approach by breaking down larger steps into smaller, reusable ones.
Document your App-as-Recipe: Ensure your App-as-Recipe is well-documented to facilitate understanding for other users. Include a description of its purpose, inputs, outputs, and dependencies.
Use descriptive names and tags: Use descriptive names that clarify the recipe’s function—leverage tags to categorize recipes, making them easier to locate later.
Use version control: Track recipe changes over time with version control. This feature allows for seamless reversion to previous versions if necessary.
Use permissions to govern access: Control access to your App-as-Recipe through permissions. Grant editing access to specific users, while allowing others to utilize the recipe in their workflows.
Know when to use App-as-Recipe vs. developing a plugin: App-as-Recipe may not be suitable for every situation. Evaluate whether a plugin better suits your needs, particularly when advanced functionality or customization is required.
Exploring App-as-Recipe Use Cases
Here are some example use cases to help you understand the difference:
Example Use Case 1: Data Preprocessing – Your team regularly preprocesses data using consistent steps, such as cleaning, encoding, and normalization. You want to standardize and reuse these steps across multiple projects.
Solution: App-as-Recipe is ideal for this use case, as it allows you to create a single, reusable recipe that can be integrated into multiple workflows, ensuring consistent preprocessing.
Example Use Case 2: Your team is working with client feedback data and needs to carry out sentiment evaluation using a specific natural language processing (NLP) technique not available in Dataiku’s built-in utilities. You want to create a reusable element that can be effortlessly integrated into your data workflows.
Solution: In this case, creating a plugin is a more suitable option. Plugins can deliver advanced features and personalization not attainable through App-as-Recipe. By creating a plugin, you can implement the custom algorithm and make it available in your data workflows.
In summary, use App-as-Recipe to package existing components for project reuse and standardization. Consider developing a plugin when you require advanced functionality or customization beyond what is available in Dataiku’s built-in tools.
Unlock the Full Potential of Dataiku's App-as-Recipe
The App-as-Recipe feature in Dataiku DSS is a potent instrument for optimizing data workflows. By packaging groups of components as a single unit, App-as-Recipe promotes reusability, consistency, and governance. This guide provided example use cases and best practices for effectively utilizing the feature in your data projects. Whether you are an experienced data scientist or a beginner, App-as-Recipe is an indispensable tool for your arsenal.
