Snowflake DSA-C02 - Questions & Answers

Explanation:
Options for Sharing in Snowflake
You can share data in Snowflake using one of the following options: a Listing, in which you offer a share and additional metadata as a data product to one or more ac-counts, a Direct Share, in which you directly share specific database objects (a share) to another account in your region, a Data Exchange, in which you set up and manage a group of accounts and offer a share to that group.

Explanation:
What is Feature Engineering?
Feature engineering is the process of transforming raw data into features that are suitable for ma-chine learning models. In other words, it is the process of selecting, extracting, and transforming the most relevant features from the available data to build more accurate and efficient machine learning models.
The success of machine learning models heavily depends on the quality of the features used to train them.
Feature engineering involves a set of techniques that enable us to create new features by combining or transforming the existing ones. These techniques help to highlight the most important pat-terns and relationships in the data, which in turn helps the machine learning model to learn from the data more effectively.
What is a Feature?
In the context of machine learning, a feature (also known as a variable or attribute) is an individual measurable property or characteristic of a data point that is used as input for a machine learning al-gorithm. Features can be numerical, categorical, or text-based, and they represent different aspects of the data that are relevant to the problem at hand. For example, in a dataset of housing prices, features could include the number of bedrooms, the square footage, the location, and the age of the property. In a dataset of customer demographics, features could include age, gender, income level, and occupation. The choice and quality of features are critical in machine learning, as they can greatly impact the ac-curacy and performance of the model.
Why do we Engineer Features?
We engineer features to improve the performance of machine learning models by providing them with relevant and informative input data. Raw data may contain noise, irrelevant information, or missing values, which can lead to inaccurate or biased model predictions. By engineering features, we can extract meaningful information from the raw data, create new variables that capture important patterns and relationships, and transform the data into a more suitable format for machine learning algorithms. Feature engineering can also help in addressing issues such as overfitting, underfitting, and high di-mensionality.
For example, by reducing the number of features, we can prevent the model from be-coming too complex or overfitting to the training data. By selecting the most relevant features, we can improve the model's accuracy and interpretability. In addition, feature engineering is a crucial step in preparing data for analysis and decision-making in various fields, such as finance, healthcare, marketing, and social sciences. It can help uncover hidden insights, identify trends and patterns, and support data-driven decision-making.
We engineer features for various reasons, and some of the main reasons include: Improve User Experience: The primary reason we engineer features is to enhance the user experience of a product or service. By adding new features, we can make the product more intuitive, efficient, and user-friendly, which can increase user satisfaction and engagement. Competitive Advantage: Another reason we engineer features is to gain a competitive advantage in the marketplace. By offering unique and innovative features, we can differentiate our product from competitors and attract more customers. Meet Customer Needs: We engineer features to meet the evolving needs of customers. By analyzing user feedback, market trends, and customer behavior, we can identify areas where new features could enhance the product's value and meet customer needs. Increase Revenue: Features can also be engineered to generate more revenue. For example, a new feature that streamlines the checkout process can increase sales, or a feature that provides additional functionality could lead to more upsells or cross-sells. Future-
Proofing: Engineering features can also be done to future-proof a product or service. By an-ticipating future trends and potential customer needs, we can develop features that ensure the product remains relevant and useful in the long term.

Processes Involved in Feature Engineering Feature engineering in Machine learning consists of mainly 5 processes: Feature Creation, Feature Transformation, Feature Extraction, Feature Selection, and Feature Scaling. It is an iterative process that requires experimentation and testing to find the best combination of features for a given problem. The success of a machine learning model largely depends on the quality of the features used in the model.
Feature Transformation
Feature Transformation is the process of transforming the featuresinto a more suitable representation for the machine learning model. This is done to ensure that the model can effectively learn from the data.
Types of Feature Transformation: Normalization: Rescaling the features to have a similar range, such as between 0 and 1, to prevent some features from dominating others.
Scaling: Rescaling the features to have a similar scale, such as having a standard deviation of 1, to make sure the model considers all features equally. Encoding: Transforming categorical features into a numerical representation. Examples are one-hot encoding and label encoding.
Transformation: Transforming the features using mathematical operations to change the distribution or scale of the features. Examples are logarithmic, square root, and reciprocal transformations.

Explanation:
To track changing data trends, create a data drift monitor that uses the training data as a baseline and the new data as a target.
Model drift and decay are concepts that describe the process during which the performance of a model deployed to production degrades on new, unseen data or the underlying assumptions about the data change.
These are important metrics to track once models are deployed toproduction. Models must be regularly re- trained on new data. This is referred to as refitting the model. This can be done either on a periodic basis, or, in an ideal scenario, retraining can be triggered when the performance of the model degrades below a
certain pre-defined threshold.

Explanation:
It will Filters the row labelled r3.

Explanation:
Data processing and analysis cannot happen without data profiling--reviewing source data for con-tent and quality. As data gets bigger and infrastructure moves to the cloud, data profiling is increasingly important.
What is data profiling?
Data profiling is the process of reviewing source data, understanding structure, content and interrelationships, and identifying potential for data projects.
Data profiling is a crucial part of: Data warehouse and business intelligence (DW/BI) projects--dataprofiling can uncover data quality issues in data sources, and what needs to be corrected in ETL.
Data conversion and migration projects--data profiling can identify data quality issues, which you can handle in scripts and data integration tools copying data from source to target. It can also un-cover new requirements for the target system.
Source system data quality projects--data profiling can highlight data which suffers from serious or numerous quality issues, and the source of the issues (e.g. user inputs, errors in interfaces, data corruption).
Data profiling involves:
Collecting descriptive statistics like min, max, count and sum.
Collecting data types, length and recurring patterns.
Tagging data with keywords, descriptions or categories.
Performing data quality assessment, risk of performing joins on the data.

Discovering metadata and assessing its accuracy.
Identifying distributions, key candidates, foreign-key candidates, functional dependencies, embedded value dependencies, and performing inter-table analysis.

Explanation:
Since the normal curve is symmetric about its mean, its skewness is zero. This is a theoretical expla-nation for mathematical proofs, you can refer to books or websites that speak on the same in detail.

Explanation:
By using ethnicity as a sensitive field, and comparing disparity between selection rates and performance metrics for each ethnicity value, you can evaluate the fairness of the model.

Explanation:
Moving Data from Spark to Snowflake
The steps for saving the contents of a DataFrame to a Snowflake table are similar to writing from Snowflake to Spark:
1. Use the write() method of the DataFrame to construct a DataFrameWriter.
2. Specify SNOWFLAKE_SOURCE_NAME using the format() method.
3. Specify the connector options using either the option() or options() method.
4. Use the dbtable option to specify the table to which data is written.
5. Use the mode() method to specify the save mode for the content.
Examples
1. df.write
2..format(SNOWFLAKE_SOURCE_NAME)
3..options(sfOptions)
4..option("dbtable", "t2")
5..mode(SaveMode.Overwrite)
6..save()

Explanation:
What is an External Function?
An external function calls code that is executed outside Snowflake. The remotely executed code is known as a remote service. Information sent to a remote service is usually relayed through a proxy service.
Snowflake stores security-related external function information in an API integration.
External Function:
An external function is a type of UDF. Unlike other UDFs, an external function does not contain its own code; instead, the external function calls code that is stored and executed outside Snowflake.
Inside Snowflake, the external function is stored as a database object that contains information that Snowflake uses to call the remote service. This stored information includes the URL of the proxy service that relays information to and from the remote service.
Remote Service:
The remotely executed code is known as a remote service. The remote service must act like a function.
For example, it must return a value. Snowflake supports scalar external functions; the remote service must return exactly one row for each row received.
Proxy Service:
Snowflake does not call a remote service directly. Instead, Snowflake calls a proxy service, which relays the data to the remote service.
The proxy service can increase security by authenticating requests to the remote service. The proxy service can support subscription-based billing for a remote service. For example, the proxy service can verify that a caller to the remote service is a paid subscriber. The proxy service also relays the response from the remote service back to Snowflake.
Examples of proxy services include: Amazon API Gateway.
Microsoft Azure API Management service.
API Integration:
An integration is a Snowflake object that provides an interface between Snowflake and third-party services. An API integration stores information, such as security information, that is needed to work with a proxy service or remote service. An API integration is created with the CREATE API INTEGRATION command. Users can write and call their own remote services, or call remote services written by third parties. These remote services can be written using any HTTP server stack,including cloud serverless compute services such as AWS Lambda.