Facebook Hacker v.8.2.0: A Comprehensive Guide for Beginners and Experts

Dependent packageExtraapache-airflow-providers-amazonamazonapache-airflow-providers-apache-beamapache.beamapache-airflow-providers-apache-cassandraapache.cassandraapache-airflow-providers-cncf-kubernetescncf.kubernetesapache-airflow-providers-common-sqlcommon.sqlapache-airflow-providers-facebookfacebookapache-airflow-providers-microsoft-azuremicrosoft.azureapache-airflow-providers-microsoft-mssqlmicrosoft.mssqlapache-airflow-providers-mysqlmysqlapache-airflow-providers-oracleoracleapache-airflow-providers-postgrespostgresapache-airflow-providers-prestoprestoapache-airflow-providers-salesforcesalesforceapache-airflow-providers-sftpsftpapache-airflow-providers-sshsshapache-airflow-providers-trinotrinoChangelog8.8.0FeaturesAdd deferrable ''GCSObjectExistenceSensorAsync'' (#28763)Support partition_columns in BaseSQLToGCSOperator (#28677)Bug Fixes'BigQueryCreateExternalTableOperator' fix field delimiter not working with csv (#28856)Fix using private _get_credentials instead of public get_credentials (#28588)Fix'GoogleCampaignManagerReportSensor' with 'QUEUED' status (#28735)Fix BigQueryColumnCheckOperator runtime error (#28796)assign "datasetReference" attribute to dataset_reference dict. by default if not already set in create_empty_dataset method of bigquery hook (#28782)8.7.0FeaturesAdd table_resource to template fields for BigQueryCreateEmptyTableOperator (#28235)Add retry param in GCSObjectExistenceSensor (#27943)Add preserveAsciiControlCharacters to src_fmt_configs (#27679)Add deferrable mode to CloudBuildCreateBuildOperator (#27783)GCSToBigQueryOperator allows autodetect None and infers schema (#28564)Improve memory usage in Dataproc deferrable operators (#28117)Push job_id in xcom for dataproc submit job op (#28639)Bug FixesFix for issue with reading schema fields for JSON files in GCSToBigQueryOperator (#28284)Fix GCSToBigQueryOperator not respecting schema_obj (#28444)Fix GCSToGCSOperator copying list of objects without wildcard (#28111)Fix: re-enable use of parameters in gcs_to_bq which had been disabled (#27961)Set bigquery ''use_legacy_sql'' param in job config correctly (#28522)MiscRemove 'pylint' messages control instructions (#28555)Remove deprecated AIPlatformConsoleLinkk from google/provider.yaml (#28449)Use object instead of array in config.yml for config template (#28417)[misc] Get rid of 'pass' statement in conditions (#27775)Change log level to DEBUG when secret not found for google secret manager (#27856)[misc] Replace XOR '^' conditions by 'exactly_one' helper in providers (#27858)8.6.0FeaturesPersist DataprocLink for workflow operators regardless of job status (#26986)Deferrable mode for BigQueryToGCSOperator (#27683)Add Export Format to Template Fields in BigQueryToGCSOperator (#27910)Bug FixesFix to read location parameter properly in BigQueryToBigQueryOperator (#27661)Bump common.sql provider to 1.3.1 (#27888)8.5.0This release of provider is only available for Airflow 2.3+ as explained in theApache Airflow providers support policy.Clean up in-line f-string concatenation (#23591)Bump pre-commit hook versions (#22887)Use new Breese for building, pulling and verifying the images. (#23104)Fix new MyPy errors in main (#22884)6.8.0FeaturesAdd autodetect arg in BQCreateExternalTable Operator (#22710)Add links for BigQuery Data Transfer (#22280)Modify transfer operators to handle more data (#22495)Create Endpoint and Model Service, Batch Prediction and Hyperparameter Tuning Jobs operators for Vertex AI service (#22088)PostgresToGoogleCloudStorageOperator - BigQuery schema type for time zone naive fields (#22536)Update secrets backends to use get_conn_value instead of get_conn_uri (#22348)Bug FixesFix the docstrings (#22497)Fix 'download_media' url in 'GoogleDisplayVideo360SDFtoGCSOperator' (#22479)Fix to 'CloudBuildRunBuildTriggerOperator' fails to find build id. (#22419)Fail ''LocalFilesystemToGCSOperator'' if src does not exist (#22772)Remove coerce_datetime usage from GCSTimeSpanFileTransformOperator (#22501)MiscRefactor: BigQuery to GCS Operator (#22506)Remove references to deprecated operators/params in PubSub operators (#22519)New design of system tests (#22311)6.7.0FeaturesAdd dataflow_default_options to templated_fields (#22367)Add 'LocalFilesystemToGoogleDriveOperator' (#22219)Add timeout and retry to the BigQueryInsertJobOperator (#22395)Bug FixesFix skipping non-GCS located jars (#22302)[FIX] typo doc of gcs operator (#22290)Fix mistakenly added install_requires for all providers (#22382)6.6.0FeaturesSupport Uploading Bigger Files to Google Drive (#22179)Change the default 'chunk_size' to a clear representation & add documentation (#22222)Add guide for DataprocInstantiateInlineWorkflowTemplateOperator (#22062)Allow for uploading metadata with GCS Hook Upload (#22058)Add Dataplex operators (#20377)MiscAdd support for ARM platform (#22127)Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)Use yaml safe load (#22091)6.5.0FeaturesAdd Looker PDT operators (#20882)Add autodetect arg to external table creation in GCSToBigQueryOperator (#21944)Add Dataproc assets/links (#21756)Add Auto ML operators for Vertex AI service (#21470)Add GoogleCalendarToGCSOperator (#20769)Make project_id argument optional in all dataproc operators (#21866)Allow templates in more DataprocUpdateClusterOperator fields (#21865)Dataflow Assets (#21639)Extract ClientInfo to module level (#21554)Datafusion assets (#21518)Dataproc metastore assets (#21267)Normalize *_conn_id parameters in BigQuery sensors (#21430)Bug FixesFix bigquery_dts parameter docstring typo (#21786)Fixed PostgresToGCSOperator fail on empty resultset for use_server_side_cursor=True (#21307)Fix multi query scenario in bigquery example DAG (#21575)MiscSupport for Python 3.10Unpin 'google-cloud-memcache' (#21912)Unpin ''pandas-gbq'' and remove unused code (#21915)Suppress hook warnings from the Bigquery transfers (#20119)6.4.0FeaturesAdd hook for integrating with Google Calendar (#20542)Add encoding parameter to 'GCSToLocalFilesystemOperator' to fix #20901 (#20919)batch as templated field in DataprocCreateBatchOperator (#20905)Make timeout Optional for wait_for_operation (#20981)Add more SQL template fields renderers (#21237)Create CustomJob and Datasets operators for Vertex AI service (#21253)Support to upload file to Google Shared Drive (#21319)(providers_google) add a location check in bigquery (#19571)Add support for BeamGoPipelineOperator (#20386)Google Cloud Composer opearators (#21251)Enable asynchronous job submission in BigQuery hook (#21385)Optionally raise an error if source file does not exist in GCSToGCSOperator (#21391)Bug FixesCloudsql import links fix. (#21199)Fix BigQueryDataTransferServiceHook.get_transfer_run() request parameter (#21293):bug: (BigQueryHook) fix compatibility with sqlalchemy engine (#19508)MiscRefactor operator links to not create ad hoc TaskInstances (#21285)6.3.0FeaturesAdd optional location to bigquery data transfer service (#15088) (#20221)Add Google Cloud Tasks how-to documentation (#20145)Added example DAG for MSSQL to Google Cloud Storage (GCS) (#19873)Support regional GKE cluster (#18966)Delete pods by default in KubernetesPodOperator (#20575)Bug FixesFixes docstring for PubSubCreateSubscriptionOperator (#20237)Fix missing get_backup method for Dataproc Metastore (#20326)BigQueryHook fix typo in run_load doc string (#19924)Fix passing the gzip compression parameter on sftp_to_gcs. (#20553)switch to follow_redirects on httpx.get call in CloudSQL provider (#20239)avoid deprecation warnings in BigQuery transfer operators (#20502)Change download_video parameter to resourceName (#20528)Fix big query to mssql/mysql transfer issues (#20001)Fix setting of project ID in ''provide_authorized_gcloud'' (#20428)MiscMove source_objects datatype check out of GCSToBigQueryOperator.__init__ (#20347)Organ