{"id":9267,"date":"2023-09-23T02:46:39","date_gmt":"2023-09-23T00:46:39","guid":{"rendered":"https:\/\/myoceane.fr\/?p=9267"},"modified":"2023-12-24T03:34:23","modified_gmt":"2023-12-24T02:34:23","slug":"rapids-support-spark-sql-with-spark-rapids","status":"publish","type":"post","link":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/","title":{"rendered":"[Rapids] Support Spark SQL with Spark Rapids"},"content":{"rendered":"<div id=\"fb-root\"><\/div>\n<p style=\"text-align: justify;\">Spark SQL \u662f\u5f88\u591a\u516c\u53f8\u63d0\u4f9b\u7d66\u5546\u696d\u667a\u6167 BI (Business Intelligence) \u7684\u5927\u6578\u64da\u4ecb\u9762\uff0c&#x1f34b;\u7238\u5728\u4ecb\u7d39\u5b8c <a href=\"https:\/\/myoceane.fr\/index.php\/application-hive-metastore-server\/\">Spark Thrift Server \u4e4b\u5f8c<\/a>\uff0c\u4f7f\u7528\u8005\u53ef\u4ee5\u900f\u904e Spark SQL \u5c0d\u81ea\u5df1\u7684\u8cc7\u6599\u505a\u64cd\u4f5c\uff0cSpark SQL \u7684\u597d\u8655\u662f\u53ef\u4ee5\u5f88\u5bb9\u6613\u53bb\u6eff\u8db3\u4f7f\u7528\u8005\u7684\u9700\u6c42\uff0c\u4f46\u662f\u5f80\u5f80\u5bb9\u6613\u9047\u5230\u56e0\u70ba\u4f7f\u7528\u8005\u57f7\u884c\u8907\u96dc Query \u5c0e\u81f4\u7684\u6548\u80fd\u4f4e\u843d\uff0c\u672c\u7bc7\u60f3\u8981\u900f\u904e\u5c0e\u5165 Spark Rapids \u914d\u5408 Spark Thrift Server \u5f15\u5165 GPU \u7684\u8cc7\u6e90\u770b\u770b\u662f\u5426\u80fd\u5920\u52a0\u901f Spark SQL \u7684\u904b\u7b97\u6548\u7387\uff1f<\/p>\n<h4><strong>Rapids Accelerator For Apache Spark<\/strong><\/h4>\n<p>\u5728 Rapids \u7684<a href=\"https:\/\/nvidia.github.io\/spark-rapids\/Getting-Started\/\">\u4ecb\u7d39\u7db2\u7ad9<\/a>\u4e2d\uff0c\u5c31\u6709\u5f37\u8abf\uff1a<\/p>\n<blockquote>\n<p style=\"text-align: justify;\">Apache Spark 3.0+ lets users provide a plugin that can replace the backend for SQL and DataFrame operations. This requires no API changes from the user. The plugin will replace SQL operations it supports with GPU accelerated versions. If an operation is not supported it will fall back to using the Spark CPU version. Note that the plugin cannot accelerate operations that manipulate RDDs directly.<\/p>\n<\/blockquote>\n<h4>Spark Configurations<\/h4>\n<p>\u4ecb\u7d39\u5e7e\u500b\u5f88\u91cd\u8981\u7684 Spark Rapid Configuration<\/p>\n<ul>\n<li>spark.executor.resource.gpu.amount\uff1a\u4e00\u500b executor \u53ef\u4ee5\u57f7\u884c\u5e7e\u500b gpu<\/li>\n<li>spark.task.resource.gpu.amount\uff1a<\/li>\n<li>spark.plugins<\/li>\n<\/ul>\n<h4>\u554f\u984c\u4e00\uff1a\u7121\u6cd5\u53d6\u5f97 Spark Executor<\/h4>\n<p>\u4e00\u958b\u59cb\u8a2d\u5b9a\u7684 Spark Configurations \u5982\u4e0b\uff0c\u7d50\u679c\u5c31\u767c\u751f\u4e86\u7121\u6cd5\u521d\u59cb\u5316 Executor \u7684\u60c5\u6cc1\u3002<\/p>\n<pre class=\"lang:bash\">spark.executor.cores 2\nspark.executor.memory 4g\nspark.executor.resource.gpu.discoveryScript \/tmp\/getGpusResources.sh\nspark.task.cpus 1\nspark.executor.resource.gpu.amount 1\nspark.task.resource.gpu.amount 0.5\nspark.rapids.memory.pinnedPool.size 2g\nspark.rapids.sql.concurrentGpuTasks 2\nspark.rapids.sql.csv.read.double.enabled true\nspark.rapids.sql.hasNans false\nspark.rapids.sql.explain ALL\nspark.plugins com.nvidia.spark.SQLPlugin<\/pre>\n<p style=\"text-align: justify;\">\u67e5\u770b\u554f\u984c\u4e4b\u5f8c\u767c\u73fe executor memory \u8a2d\u5b9a\u592a\u9ad8\u5c0e\u81f4\u4e0d\u5920\u8cc7\u6e90\u958b\u51fa\u53ef\u4ee5\u8dd1\u7684 executor\uff0c\u4e0b\u8abf\u4e4b\u5f8c\u89e3\u6c7a\uff0c\u5982\u679c executor \u90fd\u958b\u4e0d\u51fa\u4f86\u7684\u8a71\uff0cGPU \u4e5f\u5c31\u6c92\u6709\u8fa6\u6cd5\u4f7f\u7528\u3002<\/p>\n<h4>\u554f\u984c\u4e8c\uff1aSparkSQL \u56de\u50b3\u7a7a\u8868\u683c<\/h4>\n<p style=\"text-align: justify;\">\u57f7\u884c\u7c21\u55ae\u7684 SQL command \u4f8b\u5982\uff1aSELECT * FROM demo \u7d50\u679c\u62ff\u5230\u4ee5\u4e0b\u7a7a\u5b57\u4e32\uff0c\u5df2\u56de\u5831\u7d66 Spark Rapids with Ticket <a href=\"https:\/\/github.com\/NVIDIA\/spark-rapids\/issues\/9255\">https:\/\/github.com\/NVIDIA\/spark-rapids\/issues\/9255<\/a>\uff0c\u539f\u56e0\u662f\u56e0\u70ba\u4e0d\u652f\u63f4\u4f7f\u7528 columnMapping = name \u7684 DeltaTable \u8868\u683c\u3002<\/p>\n<pre class=\"lang:bash\">+----+----+\n| _c0| _c1|\n+----+----+\n|null|null|\n|null|null|\n|null|null|\n|null|null|\n|null|null|\n|null|null|\n|null|null|\n|null|null|\n+----+----+\nonly showing top 20 rows<\/pre>\n<p>\u5f8c\u7e8c\u9700\u6c42\u5df2\u7d93\u63d0\u4ea4\uff0c\u9810\u8a08\u5728 23.10 \u7248\u672c\u5c31\u53ef\u4ee5\u5229\u7528 Spark Rapids \u53bb\u8dd1\u6709\u555f\u52d5 columnMapping \u7684 DeltaTable\uff0c\u539f\u672c\u7684 Spark Rapids \u6703\u628a\u4e0d\u540c\u7684 DeltaTable \u4e00\u5f8b\u7576\u6210 ParquetFileFormat \u4f86\u8b80\u53d6\uff0c\u4e0d\u6703\u7279\u5225\u53bb\u8655\u7406 DeltaTableFileFormat\uff0c\u4f46\u662f\u5c31\u6c92\u8fa6\u6cd5\u8655\u7406 columnMapping = &#8220;name&#8221; \u7684 DeltaTable\u3002\u4fee\u6539\u5b8c\u4e4b\u5f8c\u7684\u7a0b\u5f0f\u78bc\uff1a<\/p>\n<ol>\n<li>\u5728 GpuFileSourceScanExec \u88e1\u9762\u5229\u7528 ExternalSource \u53bb\u8655\u7406 DeltaTableFileFormat \u7684\u8f38\u5165\u8cc7\u8a0a\u3002<\/li>\n<li>\u5229\u7528\u5728 ExternalSource \u88e1\u9762\u7684 DeltaProvider \u53bb\u57f7\u884ccreateMultiFileReaderFactory\u3002<\/li>\n<\/ol>\n<p>\u539f\u672c\u7684 createMultiFileReaderFactory \u6703\u56de\u50b3\u4ee5\u4e0b GpuParquetMultiFilePartitionReaderFactory<\/p>\n<pre class=\"lang:scala\">GpuParquetMultiFilePartitionReaderFactory(\n  sqlConf,\n  broadcastedHadoopConf,\n  relation.dataSchema,\n  requiredSchema,\n  readPartitionSchema,\n  pushedDownFilters.toArray,\n  rapidsConf,\n  allMetrics,\n  queryUsesInputFile,\n  alluxioPathReplacementMap)<\/pre>\n<p>\u5f8c\u4f86\u6539\u6210<\/p>\n<pre class=\"lang:scala\">GpuParquetMultiFilePartitionReaderFactory(\n  fileScan.conf,\n  broadcastedConf,\n  prepareSchema(fileScan.relation.dataSchema),\n  prepareSchema(fileScan.requiredSchema),\n  prepareSchema(fileScan.readPartitionSchema),\n  pushedFilters,\n  fileScan.rapidsConf,\n  fileScan.allMetrics,\n  fileScan.queryUsesInputFile,\n  fileScan.alluxioPathsMap)<\/pre>\n<p>\u5728\u78ba\u5b9a DeltaTable \u53ef\u4ee5\u76f4\u63a5\u900f\u904e Spark Rapids \u8b80\u53d6\u4e4b\u5f8c\uff0c\u6211\u5011\u9047\u5230\u4e86\u6548\u80fd\u7684\u554f\u984c\u3002<\/p>\n<h4>\u554f\u984c\u4e09\uff1a\u5982\u4f55\u512a\u5316 Spark Configuration \u53c3\u6578\u53ef\u4ee5\u6700\u6709\u6548\u7387\u7684\u4f7f\u7528 GPU Cluster \u7684\u8cc7\u6e90\uff1f<\/h4>\n<p style=\"text-align: justify;\">\u70ba\u4e86\u89e3\u6c7a\u9019\u500b\u554f\u984c\uff0c\u6211\u5011\u53c3\u8003\u4e86<a href=\"https:\/\/medium.com\/walmartglobaltech\/getting-started-with-apache-spark-gpu-rapids-part-i-938664771092\">\u9019\u7bc7\u6587\u7ae0<\/a>\uff0c\u88e1\u9762\u91dd\u5c0d\u4ee5\u4e0b\u7684\u8a2d\u5b9a\u505a\u4e86\u4e00\u500b\u6e05\u695a\u7684\u63cf\u8ff0\uff0c\u6700\u91cd\u8981\u7684\u6982\u5ff5\u5c31\u662f\u8981\u8b93 GPU parallelism \u53ef\u4ee5\u7b49\u65bc CPU parallelism\u3002<\/p>\n<h4>Profiling Tool<\/h4>\n<p>\u5f8c\u7e8c\u6211\u5011\u5c07\u7e7c\u7e8c\u6e2c\u8a66\u4e0d\u540c SparkSQL \u4f7f\u7528 GPU \u7684\u60c5\u6cc1\u4e26\u4e14\u5229\u7528 Spark Rapids \u7684 Profiling Tool \u4f86\u5354\u52a9\u89c0\u5bdf\uff0c\u95dc\u65bc <a href=\"https:\/\/nvidia.github.io\/spark-rapids\/docs\/spark-profiling-tool.html\">Spark Rapids Profiling Tool<\/a> \u8a73\u7d30\u53ef\u4ee5\u53c3\u8003\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Spark SQL \u662f\u5f88\u591a\u516c\u53f8\u63d0\u4f9b\u7d66\u5546\u696d\u667a\u6167 BI (Business Intelligence) \u7684\u5927\u6578\u64da\u4ecb\u9762\uff0c&#x1f34b;\u7238\u5728\u4ecb\u7d39\u5b8c Spark Thrift Server \u4e4b\u5f8c\uff0c\u4f7f\u7528\u8005\u53ef\u4ee5\u900f\u904e Spark SQL \u5c0d\u81ea\u5df1\u7684\u8cc7\u6599\u505a\u64cd\u4f5c\uff0cSpark SQL \u7684\u597d\u8655\u662f\u53ef\u4ee5\u5f88\u5bb9\u6613\u53bb\u6eff\u8db3\u4f7f\u7528\u8005\u7684\u9700\u6c42\uff0c\u4f46\u662f\u5f80\u5f80\u5bb9\u6613\u9047\u5230\u56e0\u70ba\u4f7f\u7528\u8005\u57f7\u884c\u8907\u96dc Query \u5c0e\u81f4\u7684\u6548\u80fd\u4f4e\u843d\uff0c\u672c\u7bc7\u60f3\u8981\u900f\u904e\u5c0e\u5165 Spark Rapids \u914d\u5408 Spark Thrift Server \u5f15\u5165 GPU \u7684\u8cc7\u6e90\u770b\u770b\u662f\u5426\u80fd\u5920\u52a0\u901f Spark SQL \u7684\u904b\u7b97\u6548\u7387\uff1f<\/p>\n","protected":false},"author":1,"featured_media":8700,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[1728,1729,152,1725],"class_list":["post-9267","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-ml","tag-delta-table","tag-rapids","tag-spark","tag-sparksql"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane\" \/>\n<meta property=\"og:description\" content=\"Spark SQL \u662f\u5f88\u591a\u516c\u53f8\u63d0\u4f9b\u7d66\u5546\u696d\u667a\u6167 BI (Business Intelligence) \u7684\u5927\u6578\u64da\u4ecb\u9762\uff0c&#x1f34b;\u7238\u5728\u4ecb\u7d39\u5b8c Spark Thrift Server \u4e4b\u5f8c\uff0c\u4f7f\u7528\u8005\u53ef\u4ee5\u900f\u904e Spark SQL \u5c0d\u81ea\u5df1\u7684\u8cc7\u6599\u505a\u64cd\u4f5c\uff0cSpark SQL \u7684\u597d\u8655\u662f\u53ef\u4ee5\u5f88\u5bb9\u6613\u53bb\u6eff\u8db3\u4f7f\u7528\u8005\u7684\u9700\u6c42\uff0c\u4f46\u662f\u5f80\u5f80\u5bb9\u6613\u9047\u5230\u56e0\u70ba\u4f7f\u7528\u8005\u57f7\u884c\u8907\u96dc Query \u5c0e\u81f4\u7684\u6548\u80fd\u4f4e\u843d\uff0c\u672c\u7bc7\u60f3\u8981\u900f\u904e\u5c0e\u5165 Spark Rapids \u914d\u5408 Spark Thrift Server \u5f15\u5165 GPU \u7684\u8cc7\u6e90\u770b\u770b\u662f\u5426\u80fd\u5920\u52a0\u901f Spark SQL \u7684\u904b\u7b97\u6548\u7387\uff1f\" \/>\n<meta property=\"og:url\" content=\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\" \/>\n<meta property=\"og:site_name\" content=\"\u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane\" \/>\n<meta property=\"article:published_time\" content=\"2023-09-23T00:46:39+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-24T02:34:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png\" \/>\n\t<meta property=\"og:image:width\" content=\"936\" \/>\n\t<meta property=\"og:image:height\" content=\"248\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"\u6ab8\u6aac\u7238\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u6ab8\u6aac\u7238\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\"},\"author\":{\"name\":\"\u6ab8\u6aac\u7238\",\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b\"},\"headline\":\"[Rapids] Support Spark SQL with Spark Rapids\",\"datePublished\":\"2023-09-23T00:46:39+00:00\",\"dateModified\":\"2023-12-24T02:34:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\"},\"wordCount\":192,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b\"},\"image\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png\",\"keywords\":[\"Delta Table\",\"Rapids\",\"Spark\",\"SparkSQL\"],\"articleSection\":[\"Big Data &amp; Machine Learning\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\",\"url\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\",\"name\":\"[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane\",\"isPartOf\":{\"@id\":\"https:\/\/myoceane.fr\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png\",\"datePublished\":\"2023-09-23T00:46:39+00:00\",\"dateModified\":\"2023-12-24T02:34:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage\",\"url\":\"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png\",\"contentUrl\":\"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png\",\"width\":936,\"height\":248},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/myoceane.fr\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Rapids] Support Spark SQL with Spark Rapids\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/myoceane.fr\/#website\",\"url\":\"https:\/\/myoceane.fr\/\",\"name\":\"M-Y-Oceane \u60f3\u65b9\u6d89\u6cd5\u3002\u91cf\u74f6\u5916\u7684\u5929\u7a7a\",\"description\":\"\u60f3\u65b9\u6d89\u6cd5, France, Taiwan, Health, Information Technology\",\"publisher\":{\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/myoceane.fr\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b\",\"name\":\"\u6ab8\u6aac\u7238\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/6cc678684664f8ad45a8d56a6630b183?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/6cc678684664f8ad45a8d56a6630b183?s=96&d=mm&r=g\",\"caption\":\"\u6ab8\u6aac\u7238\"},\"logo\":{\"@id\":\"https:\/\/myoceane.fr\/#\/schema\/person\/image\/\"},\"url\":\"https:\/\/myoceane.fr\/index.php\/author\/johnny5584767gmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/","og_locale":"en_US","og_type":"article","og_title":"[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane","og_description":"Spark SQL \u662f\u5f88\u591a\u516c\u53f8\u63d0\u4f9b\u7d66\u5546\u696d\u667a\u6167 BI (Business Intelligence) \u7684\u5927\u6578\u64da\u4ecb\u9762\uff0c&#x1f34b;\u7238\u5728\u4ecb\u7d39\u5b8c Spark Thrift Server \u4e4b\u5f8c\uff0c\u4f7f\u7528\u8005\u53ef\u4ee5\u900f\u904e Spark SQL \u5c0d\u81ea\u5df1\u7684\u8cc7\u6599\u505a\u64cd\u4f5c\uff0cSpark SQL \u7684\u597d\u8655\u662f\u53ef\u4ee5\u5f88\u5bb9\u6613\u53bb\u6eff\u8db3\u4f7f\u7528\u8005\u7684\u9700\u6c42\uff0c\u4f46\u662f\u5f80\u5f80\u5bb9\u6613\u9047\u5230\u56e0\u70ba\u4f7f\u7528\u8005\u57f7\u884c\u8907\u96dc Query \u5c0e\u81f4\u7684\u6548\u80fd\u4f4e\u843d\uff0c\u672c\u7bc7\u60f3\u8981\u900f\u904e\u5c0e\u5165 Spark Rapids \u914d\u5408 Spark Thrift Server \u5f15\u5165 GPU \u7684\u8cc7\u6e90\u770b\u770b\u662f\u5426\u80fd\u5920\u52a0\u901f Spark SQL \u7684\u904b\u7b97\u6548\u7387\uff1f","og_url":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/","og_site_name":"\u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane","article_published_time":"2023-09-23T00:46:39+00:00","article_modified_time":"2023-12-24T02:34:23+00:00","og_image":[{"width":936,"height":248,"url":"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png","type":"image\/png"}],"author":"\u6ab8\u6aac\u7238","twitter_card":"summary_large_image","twitter_misc":{"Written by":"\u6ab8\u6aac\u7238","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#article","isPartOf":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/"},"author":{"name":"\u6ab8\u6aac\u7238","@id":"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b"},"headline":"[Rapids] Support Spark SQL with Spark Rapids","datePublished":"2023-09-23T00:46:39+00:00","dateModified":"2023-12-24T02:34:23+00:00","mainEntityOfPage":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/"},"wordCount":192,"commentCount":1,"publisher":{"@id":"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b"},"image":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage"},"thumbnailUrl":"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png","keywords":["Delta Table","Rapids","Spark","SparkSQL"],"articleSection":["Big Data &amp; Machine Learning"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/","url":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/","name":"[Rapids] Support Spark SQL with Spark Rapids - \u60f3\u65b9\u6d89\u6cd5 - \u91cf\u74f6\u5916\u7684\u5929\u7a7a M-Y-Oceane","isPartOf":{"@id":"https:\/\/myoceane.fr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage"},"image":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage"},"thumbnailUrl":"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png","datePublished":"2023-09-23T00:46:39+00:00","dateModified":"2023-12-24T02:34:23+00:00","breadcrumb":{"@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#primaryimage","url":"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png","contentUrl":"https:\/\/myoceane.fr\/wp-content\/uploads\/2022\/08\/RAPIDSSpark.png","width":936,"height":248},{"@type":"BreadcrumbList","@id":"https:\/\/myoceane.fr\/index.php\/rapids-support-spark-sql-with-spark-rapids\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/myoceane.fr\/"},{"@type":"ListItem","position":2,"name":"[Rapids] Support Spark SQL with Spark Rapids"}]},{"@type":"WebSite","@id":"https:\/\/myoceane.fr\/#website","url":"https:\/\/myoceane.fr\/","name":"M-Y-Oceane \u60f3\u65b9\u6d89\u6cd5\u3002\u91cf\u74f6\u5916\u7684\u5929\u7a7a","description":"\u60f3\u65b9\u6d89\u6cd5, France, Taiwan, Health, Information Technology","publisher":{"@id":"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/myoceane.fr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/myoceane.fr\/#\/schema\/person\/4a4552fb8c27693083d465e12db7658b","name":"\u6ab8\u6aac\u7238","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/myoceane.fr\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/6cc678684664f8ad45a8d56a6630b183?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6cc678684664f8ad45a8d56a6630b183?s=96&d=mm&r=g","caption":"\u6ab8\u6aac\u7238"},"logo":{"@id":"https:\/\/myoceane.fr\/#\/schema\/person\/image\/"},"url":"https:\/\/myoceane.fr\/index.php\/author\/johnny5584767gmail-com\/"}]}},"amp_enabled":false,"_links":{"self":[{"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/posts\/9267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/comments?post=9267"}],"version-history":[{"count":20,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/posts\/9267\/revisions"}],"predecessor-version":[{"id":9495,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/posts\/9267\/revisions\/9495"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/media\/8700"}],"wp:attachment":[{"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/media?parent=9267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/categories?post=9267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/myoceane.fr\/index.php\/wp-json\/wp\/v2\/tags?post=9267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}