Business intelligence made simple
Fixed quoting issue with variables - fixes #391
Andrew Kanef49fbfed7b9e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ca54e23f..84f671ba 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,7 @@-## 2.5.1 (unreleased)+## 2.6.0 (unreleased)+- Fixed quoting issue with variables+- Custom adapters now need to specify how to quote variables in queries- Added experimental support for Propshaft- Fixed error with empty results with InfluxDBdiff --git a/README.md b/README.mdindex 88b69a12..2914eb34 100644--- a/README.md+++ b/README.md@@ -61,7 +61,7 @@ For production, specify your database:ENV["BLAZER_DATABASE_URL"] = "postgres://user:password@hostname:5432/database"```-Blazer tries to protect against queries which modify data (by running each query in a transaction and rolling it back), but a safer approach is to use a read-only user. [See how to create one](#permissions).+When possible, Blazer tries to protect against queries which modify data by running each query in a transaction and rolling it back, but a safer approach is to use a read-only user. [See how to create one](#permissions).#### Checks (optional)@@ -144,11 +144,9 @@ Be sure to render or redirect for unauthorized users.## Permissions-Blazer runs each query in a transaction and rolls it back to prevent queries from modifying data. As an additional line of defense, we recommend using a read only user.-### PostgreSQL-Create a user with read only permissions:+Create a user with read-only permissions:```sqlBEGIN;@@ -162,7 +160,7 @@ COMMIT;### MySQL-Create a user with read only permissions:+Create a user with read-only permissions:```sqlGRANT SELECT, SHOW VIEW ON database_name.* TO blazer@’127.0.0.1′ IDENTIFIED BY ‘secret123‘;@@ -171,7 +169,7 @@ FLUSH PRIVILEGES;### MongoDB-Create a user with read only permissions:+Create a user with read-only permissions:```db.createUser({user: "blazer", pwd: "password", roles: ["read"]})@@ -656,6 +654,8 @@ data_sources:url: redshift://user:password@hostname:5439/database```+Use a [read-only user](https://docs.aws.amazon.com/redshift/latest/dg/r_GRANT.html).+### Apache DrillAdd [drill-sergeant](https://github.com/ankane/drill-sergeant) to your Gemfile and set:@@ -667,6 +667,8 @@ data_sources:url: http://hostname:8047```+Use a [read-only user](https://drill.apache.org/docs/roles-and-privileges/).+### Apache HiveAdd [hexspace](https://github.com/ankane/hexspace) to your Gemfile and set:@@ -690,6 +692,8 @@ data_sources:url: ignite://user:password@hostname:10800```+Use a [read-only user](https://www.gridgain.com/docs/latest/administrators-guide/security/authorization-permissions) (requires a third-party plugin).+### Apache SparkAdd [hexspace](https://github.com/ankane/hexspace) to your Gemfile and set:@@ -713,6 +717,8 @@ data_sources:url: cassandra://user:password@hostname:9042/keyspace```+Use a [read-only role](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useSecurePermission.html).+### DruidEnable [SQL support](http://druid.io/docs/latest/querying/sql.html#configuration) on the broker and set:@@ -724,6 +730,8 @@ data_sources:url: http://hostname:8082```+Use a [read-only role](https://druid.apache.org/docs/latest/development/extensions-core/druid-basic-security.html).+### ElasticsearchAdd [elasticsearch](https://github.com/elastic/elasticsearch-ruby) to your Gemfile and set:@@ -735,6 +743,8 @@ data_sources:url: http://user:password@hostname:9200```+Use a [read-only role](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html).+### Google BigQueryAdd [google-cloud-bigquery](https://github.com/GoogleCloudPlatform/google-cloud-ruby/tree/master/google-cloud-bigquery) to your Gemfile and set:@@ -757,6 +767,8 @@ data_sources:url: ibm-db://user:password@hostname:50000/database```+Use a [read-only user](https://www.ibm.com/support/pages/creating-read-only-database-permissions-user).+### InfluxDBAdd [influxdb](https://github.com/influxdata/influxdb-ruby) to your Gemfile and set:@@ -768,7 +780,7 @@ data_sources:url: http://user:password@hostname:8086/database```-Supports [InfluxQL](https://docs.influxdata.com/influxdb/v1.8/query_language/explore-data/)+Use a [read-only user](https://docs.influxdata.com/influxdb/v1.8/administration/authentication_and_authorization/). Supports [InfluxQL](https://docs.influxdata.com/influxdb/v1.8/query_language/explore-data/).### MongoDB@@ -782,6 +794,8 @@ data_sources:url: mongodb://user:password@hostname:27017/database```+Use a [read-only user](#mongodb).+### MySQLAdd [mysql2](https://github.com/brianmario/mysql2) to your Gemfile (if it’s not there) and set:@@ -792,6 +806,8 @@ data_sources:url: mysql2://user:password@hostname:3306/database```+Use a [read-only user](#mysql).+### Neo4jAdd [neo4j-core](https://github.com/neo4jrb/neo4j-core) to your Gemfile and set:@@ -803,6 +819,8 @@ data_sources:url: http://user:password@hostname:7474```+Use a [read-only user](https://neo4j.com/docs/cypher-manual/current/access-control/manage-privileges/).+### OpenSearchAdd [opensearch-ruby](https://github.com/opensearch-project/opensearch-ruby) to your Gemfile and set:@@ -814,6 +832,8 @@ data_sources:url: http://user:password@hostname:9200```+Use a [read-only user](https://opensearch.org/docs/latest/security-plugin/access-control/permissions/).+### OracleAdd [activerecord-oracle_enhanced-adapter](https://github.com/rsim/oracle-enhanced) and [ruby-oci8](https://github.com/kubo/ruby-oci8) to your Gemfile and set:@@ -824,6 +844,8 @@ data_sources:url: oracle-enhanced://user:password@hostname:1521/database```+Use a [read-only user](https://docs.oracle.com/cd/B19306_01/network.102/b14266/authoriz.htm).+### PostgreSQLAdd [pg](https://github.com/ged/ruby-pg) to your Gemfile (if it’s not there) and set:@@ -834,6 +856,8 @@ data_sources:url: postgres://user:password@hostname:5432/database```+Use a [read-only user](#postgresql).+### PrestoAdd [presto-client](https://github.com/treasure-data/presto-client-ruby) to your Gemfile and set:@@ -844,6 +868,8 @@ data_sources:url: presto://user@hostname:8080/catalog```+Use a [read-only user](https://prestodb.io/docs/current/security/built-in-system-access-control.html).+### SalesforceAdd [restforce](https://github.com/restforce/restforce) to your Gemfile and set:@@ -865,7 +891,7 @@ SALESFORCE_CLIENT_SECRET="client secret"SALESFORCE_API_VERSION="41.0"```-Supports [SOQL](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql.htm)+Use a read-only user. Supports [SOQL](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql.htm).### Socrata Open Data API (SODA)@@ -879,7 +905,7 @@ data_sources:app_token: ...```-Supports [SoQL](https://dev.socrata.com/docs/functions/)+Supports [SoQL](https://dev.socrata.com/docs/functions/).### Snowflake@@ -913,6 +939,8 @@ data_sources:conn_str: Driver=/path/to/libSnowflake.so;uid=user;pwd=password;server=host.snowflakecomputing.com```+Use a [read-only role](https://docs.snowflake.com/en/user-guide/security-access-control-configure.html).+### SQLiteAdd [sqlite3](https://github.com/sparklemotion/sqlite3-ruby) to your Gemfile and set:@@ -933,6 +961,8 @@ data_sources:url: sqlserver://user:password@hostname:1433/database```+Use a [read-only user](https://docs.microsoft.com/en-us/sql/relational-databases/security/authentication-access/getting-started-with-database-engine-permissions?view=sql-server-ver15).+## Creating an AdapterCreate an adapter for any data store with:@@ -1009,6 +1039,22 @@ override_csp: true## Upgrading+### 2.6++Custom adapters now need to specify how to quote variables in queries (there is no longer a default)++```ruby+class FooAdapter < Blazer::Adapters::BaseAdapter+ def quoting+ :backslash_escape # single quote strings and convert ' to \' and \ to \\+ # or+ :single_quote_escape # single quote strings and convert ' to ''+ # or+ ->(value) { ... } # custom method+ end+end+```+### 2.3To archive queries, create a migrationdiff --git a/app/controllers/blazer/base_controller.rb b/app/controllers/blazer/base_controller.rbindex 1b359544..fc22511b 100644--- a/app/controllers/blazer/base_controller.rb+++ b/app/controllers/blazer/base_controller.rb@@ -32,46 +32,32 @@ module Blazerprivate- def process_vars(statement, data_source, var_params = nil)+ def process_vars(statement, var_params = nil)var_params ||= request.query_parameters- (@bind_vars ||= []).concat(Blazer.extract_vars(statement)).uniq!+ (@bind_vars ||= []).concat(statement.variables).uniq!# update in-place so populated in view and consistent across queries on dashboard@bind_vars.each do |var|- var_params[var] ||= Blazer.data_sources[data_source].variable_defaults[var]- end- @success = @bind_vars.all? { |v| var_params[v] }-- if @success- @bind_vars.each do |var|- value = var_params[var].presence- if value- if ["start_time", "end_time"].include?(var)- value = value.to_s.gsub(" ", "+") # fix for Quip bug- end-- if var.end_with?("_at")- begin- value = Blazer.time_zone.parse(value)- rescue- # do nothing- end- end-- if value =~ /\A\d+\z/- value = value.to_i- elsif value =~ /\A\d+\.\d+\z/- value = value.to_f- end- end- value = Blazer.transform_variable.call(var, value) if Blazer.transform_variable- statement.gsub!("{#{var}}", ActiveRecord::Base.connection.quote(value))+ if !var_params[var]+ default = statement.data_source.variable_defaults[var]+ # only add if default exists+ var_params[var] = default if defaultendend+ runnable = @bind_vars.all? { |v| var_params[v] }+ statement.add_values(var_params) if runnable+ runnable+ end++ def refresh_query(query)+ statement = query.statement_object+ runnable = process_vars(statement)+ cohort_analysis_statement(statement) if statement.cohort_analysis?+ statement.clear_cache if runnableenddef add_cohort_analysis_vars@bind_vars << "cohort_period" unless @bind_vars.include?("cohort_period")- @smart_vars["cohort_period"] = ["day", "week", "month"]+ @smart_vars["cohort_period"] = ["day", "week", "month"] if @smart_vars# TODO create var_params methodrequest.query_parameters["cohort_period"] ||= "week"end@@ -97,6 +83,25 @@ module Blazer[smart_var, error]end+ def cohort_analysis_statement(statement)+ @cohort_period = params["cohort_period"] || "week"+ @cohort_period = "week" unless ["day", "week", "month"].include?(@cohort_period)++ # for now+ @conversion_period = @cohort_period+ @cohort_days =+ case @cohort_period+ when "day"+ 1+ when "week"+ 7+ when "month"+ 30+ end++ statement.apply_cohort_analysis(period: @cohort_period, days: @cohort_days)+ end+# TODO allow all keys# or show error message for disallowed keysUNPERMITTED_KEYS = [:controller, :action, :id, :host, :query, :dashboard, :query_id, :query_ids, :table_names, :authenticity_token, :utf8, :_method, :commit, :statement, :data_source, :name, :fork_query_id, :blazer, :run_id, :script_name, :original_script_name]diff --git a/app/controllers/blazer/dashboards_controller.rb b/app/controllers/blazer/dashboards_controller.rbindex c752a8cc..5cb5564d 100644--- a/app/controllers/blazer/dashboards_controller.rb+++ b/app/controllers/blazer/dashboards_controller.rb@@ -21,11 +21,8 @@ module Blazerdef show@queries = @dashboard.dashboard_queries.order(:position).preload(:query).map(&:query)- @statements = []@queries.each do |query|- statement = query.statement.dup- process_vars(statement, query.data_source)- @statements << statement+ @success = process_vars(query.statement_object)end@bind_vars ||= []@@ -61,11 +58,7 @@ module Blazerdef refresh@dashboard.queries.each do |query|- data_source = Blazer.data_sources[query.data_source]- statement = query.statement.dup- process_vars(statement, query.data_source)- Blazer.transform_statement.call(data_source, statement) if Blazer.transform_statement- data_source.clear_cache(statement)+ refresh_query(query)endredirect_to dashboard_path(@dashboard, params: variable_params(@dashboard))enddiff --git a/app/controllers/blazer/queries_controller.rb b/app/controllers/blazer/queries_controller.rbindex d980747a..04e4c0a7 100644--- a/app/controllers/blazer/queries_controller.rb+++ b/app/controllers/blazer/queries_controller.rb@@ -59,22 +59,19 @@ module Blazerenddef show- @statement = @query.statement.dup- process_vars(@statement, @query.data_source)+ @statement = @query.statement_object+ @success = process_vars(@statement)@smart_vars = {}@sql_errors = []- data_source = Blazer.data_sources[@query.data_source]@bind_vars.each do |var|- smart_var, error = parse_smart_variables(var, data_source)+ smart_var, error = parse_smart_variables(var, @statement.data_source)@smart_vars[var] = smart_var if smart_var@sql_errors << error if errorend@query.update!(status: "active") if @query.respond_to?(:status) && @query.status.in?(["archived", nil])- Blazer.transform_statement.call(data_source, @statement) if Blazer.transform_statement-add_cohort_analysis_vars if @query.cohort_analysis?end@@ -90,12 +87,13 @@ module Blazerdata_source ||= params[:data_source]@data_source = Blazer.data_sources[data_source]- @statement = params[:statement]+ @statement = Blazer::Statement.new(params[:statement], @data_source)# before process_vars- @cohort_analysis = Query.new(statement: @statement).cohort_analysis?+ @cohort_analysis = @statement.cohort_analysis?+# fallback for now for users with open tabs@var_params = request.request_parameters["variables"] || request.request_parameters- process_vars(@statement, data_source, @var_params)+ @success = process_vars(@statement, @var_params)@only_chart = params[:only_chart]@run_id = blazer_params[:run_id]@@ -134,7 +132,7 @@ module Blazeroptions = {user: blazer_user, query: @query, refresh_cache: params[:check], run_id: @run_id, async: Blazer.async}if Blazer.async && request.format.symbol != :csv- Blazer::RunStatementJob.perform_later(@data_source.id, @statement, options)+ Blazer::RunStatementJob.perform_later(@data_source.id, @statement.statement, options.merge(values: @statement.values))wait_start = Time.nowloop dosleep(0.1)@@ -142,7 +140,7 @@ module Blazerbreak if @result || Time.now - wait_start > 3endelse- @result = Blazer::RunStatement.new.perform(@data_source, @statement, options)+ @result = Blazer::RunStatement.new.perform(@statement, options)endif @result@@ -172,12 +170,7 @@ module Blazerenddef refresh- data_source = Blazer.data_sources[@query.data_source]- @statement = @query.statement.dup- process_vars(@statement, @query.data_source)- Blazer.transform_statement.call(data_source, @statement) if Blazer.transform_statement- @statement = cohort_analysis_statement(data_source, @statement) if @query.cohort_analysis?- data_source.clear_cache(@statement)+ refresh_query(@query)redirect_to query_path(@query, params: variable_params(@query))end@@ -288,6 +281,9 @@ module Blazerrender layout: falseendformat.csv do+ # not ideal, but useful for testing+ raise Error, @error if @error && Rails.env.test?+send_data csv_data(@columns, @rows, @data_source), type: "text/csv; charset=utf-8; header=present", disposition: "attachment; filename=\"#{@query.try(:name).try(:parameterize).presence || 'query'}.csv\""endend@@ -369,34 +365,12 @@ module Blazerenddef run_cohort_analysis- unless @data_source.supports_cohort_analysis?+ unless @statement.data_source.supports_cohort_analysis?@cohort_error = "This data source does not support cohort analysis"end@show_cohort_rows = !params[:query_id] || @cohort_error-- unless @show_cohort_rows- @statement = cohort_analysis_statement(@data_source, @statement)- end- end-- def cohort_analysis_statement(data_source, statement)- @cohort_period = params["cohort_period"] || "week"- @cohort_period = "week" unless ["day", "week", "month"].include?(@cohort_period)-- # for now- @conversion_period = @cohort_period- @cohort_days =- case @cohort_period- when "day"- 1- when "week"- 7- when "month"- 30- end-- data_source.cohort_analysis_statement(statement, period: @cohort_period, days: @cohort_days)+ cohort_analysis_statement(@statement) unless @show_cohort_rowsenddef render_cohort_analysisdiff --git a/app/models/blazer/query.rb b/app/models/blazer/query.rbindex 0605f3e5..f51fd8f9 100644--- a/app/models/blazer/query.rb+++ b/app/models/blazer/query.rb@@ -35,13 +35,19 @@ module Blazerenddef variables- variables = Blazer.extract_vars(statement)+ # don't require data_source to be loaded+ variables = Statement.new(statement).variablesvariables += ["cohort_period"] if cohort_analysis?variablesenddef cohort_analysis?- /\/\*\s*cohort analysis\s*\*\//i.match?(statement)+ # don't require data_source to be loaded+ Statement.new(statement).cohort_analysis?+ end++ def statement_object+ Statement.new(statement, data_source)endendenddiff --git a/app/views/blazer/dashboards/show.html.erb b/app/views/blazer/dashboards/show.html.erbindex d93494ed..4afbfc60 100644--- a/app/views/blazer/dashboards/show.html.erb+++ b/app/views/blazer/dashboards/show.html.erb@@ -39,7 +39,7 @@</div></div><script>- <%= blazer_js_var "data", {statement: @statements[i], query_id: query.id, data_source: query.data_source, only_chart: true, cohort_period: params[:cohort_period]} %>+ <%= blazer_js_var "data", {statement: query.statement, query_id: query.id, variables: variable_params(query), only_chart: true, cohort_period: params[:cohort_period]} %>runQuery(data, function (data) {$("#chart-<%= i %>").html(data)diff --git a/app/views/blazer/queries/show.html.erb b/app/views/blazer/queries/show.html.erbindex 0a80a0d4..d7c74a2c 100644--- a/app/views/blazer/queries/show.html.erb+++ b/app/views/blazer/queries/show.html.erb@@ -1,5 +1,11 @@<% blazer_title @query.name %>+<% if @success %>+ <% run_data = {statement: @query.statement, query_id: @query.id, variables: variable_params(@query)} %>+ <% run_data.merge!(forecast: "t") if params[:forecast] %>+ <% run_data.merge!(cohort_period: params[:cohort_period]) if params[:cohort_period] %>+<% end %>+<div class="topbar"><div class="container"><div class="row" style="padding-top: 13px;">@@ -14,7 +20,7 @@<%= link_to "Fork", new_query_path(params: variable_params(@query).merge(fork_query_id: @query.id, data_source: @query.data_source, name: @query.name)), class: "btn btn-info" %><% if !@error && @success %>- <%= button_to "Download", run_queries_path(query_id: @query.id, format: "csv", forecast: params[:forecast], cohort_period: params[:cohort_period]), params: {statement: @statement}, class: "btn btn-primary" %>+ <%= button_to "Download", run_queries_path(format: "csv"), params: run_data, class: "btn btn-primary" %><% end %></div></div>@@ -39,7 +45,7 @@<%= render partial: "blazer/variables", locals: {action: query_path(@query)} %>-<pre id="code"><code><%= @statement %></code></pre>+<pre id="code"><code><%= @statement.display_statement %></code></pre><% if @success %><div id="results">@@ -56,9 +62,7 @@$("#results").addClass("query-error").html(message)}- <% data = {statement: @statement, query_id: @query.id, data_source: @query.data_source, variables: variable_params(@query)} %>- <% data.merge!(forecast: "t") if params[:forecast] %>- <%= blazer_js_var "data", data %>+ <%= blazer_js_var "data", run_data %>runQuery(data, showRun, showError)</script>diff --git a/lib/blazer.rb b/lib/blazer.rbindex 10a69e79..3c6eaf90 100644--- a/lib/blazer.rb+++ b/lib/blazer.rb@@ -1,14 +1,18 @@# dependencies-require "csv"-require "yaml"require "chartkick"require "safely/core"+# stdlib+require "csv"+require "json"+require "yaml"+# modulesrequire "blazer/version"require "blazer/data_source"require "blazer/result"require "blazer/run_statement"+require "blazer/statement"# adaptersrequire "blazer/adapters/base_adapter"@@ -78,6 +82,7 @@ module Blazerself.images = falseself.override_csp = false+ VARIABLE_MESSAGE = "Variable cannot be used in this position"TIMEOUT_MESSAGE = "Query timed out :("TIMEOUT_ERRORS = ["canceling statement due to statement timeout", # postgres@@ -130,6 +135,7 @@ module Blazerendend+ # TODO move to Statement and remove in 3.0.0def self.extract_vars(statement)# strip commented out lines# and regex {1} or {1,2}@@ -150,9 +156,8 @@ module BlazerActiveSupport::Notifications.instrument("run_check.blazer", check_id: check.id, query_id: check.query.id, state_was: check.state) do |instrument|# try 3 times on timeout errors- data_source = data_sources[check.query.data_source]- statement = check.query.statement- Blazer.transform_statement.call(data_source, statement) if Blazer.transform_statement+ statement = check.query.statement_object+ data_source = statement.data_sourcewhile tries <= 3result = data_source.run_statement(statement, refresh_cache: true, check: check, query: check.query)@@ -180,7 +185,8 @@ module Blazer# TODO use proper logfmtRails.logger.info "[blazer check] query=#{check.query.name} state=#{check.state} rows=#{result.rows.try(:size)} error=#{result.error}"- instrument[:statement] = statement+ # should be no variables+ instrument[:statement] = statement.bind_statementinstrument[:data_source] = data_sourceinstrument[:state] = check.stateinstrument[:rows] = result.rows.try(:size)diff --git a/lib/blazer/adapters/athena_adapter.rb b/lib/blazer/adapters/athena_adapter.rbindex 115564c9..c8b80b93 100644--- a/lib/blazer/adapters/athena_adapter.rb+++ b/lib/blazer/adapters/athena_adapter.rb@@ -1,31 +1,50 @@module Blazermodule Adaptersclass AthenaAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params = [])require "digest/md5"columns = []rows = []error = nil- query_options = {- query_string: statement,- # use token so we fetch cached results after query is run- client_request_token: Digest::MD5.hexdigest([statement, data_source.id, settings["workgroup"]].compact.join("/")),- query_execution_context: {- database: database,+ begin+ # use empty? since any? doesn't work for [nil]+ if !bind_params.empty?+ request_token = Digest::MD5.hexdigest([statement, bind_params.to_json, data_source.id, settings["workgroup"]].compact.join("/"))+ statement_name = "blazer_#{request_token}"+ begin+ client.create_prepared_statement({+ statement_name: statement_name,+ work_group: settings["workgroup"],+ query_statement: statement+ })+ rescue Aws::Athena::Errors::InvalidRequestException => e+ raise e unless e.message.include?("already exists in WorkGroup")+ end+ using_statement = bind_params.map { |v| data_source.quote(v) }.join(", ")+ statement = "EXECUTE #{statement_name} USING #{using_statement}"+ else+ request_token = Digest::MD5.hexdigest([statement, data_source.id, settings["workgroup"]].compact.join("/"))+ end++ query_options = {+ query_string: statement,+ # use token so we fetch cached results after query is run+ client_request_token: request_token,+ query_execution_context: {+ database: database,+ }}- }- if settings["output_location"]- query_options[:result_configuration] = {output_location: settings["output_location"]}- end+ if settings["output_location"]+ query_options[:result_configuration] = {output_location: settings["output_location"]}+ end- if settings["workgroup"]- query_options[:work_group] = settings["workgroup"]- end+ if settings["workgroup"]+ query_options[:work_group] = settings["workgroup"]+ end- beginresp = client.start_query_execution(**query_options)query_execution_id = resp.query_execution_id@@ -111,12 +130,29 @@ module Blazer"SELECT * FROM {table} LIMIT 10"end+ # https://docs.aws.amazon.com/athena/latest/ug/select.html#select-escaping+ def quoting+ :single_quote_escape+ end++ # https://docs.aws.amazon.com/athena/latest/ug/querying-with-prepared-statements.html+ def parameter_binding+ engine_version > 1 ? :positional : nil+ end+privatedef database@database ||= settings["database"] || "default"end+ # note: this setting is experimental+ # it does *not* need to be set to use engine version 2+ # prepared statements must be manually deleted if enabled+ def engine_version+ @engine_version ||= (settings["engine_version"] || 1).to_i+ end+def fetch_error(query_execution_id)client.get_query_execution(query_execution_id: query_execution_iddiff --git a/lib/blazer/adapters/base_adapter.rb b/lib/blazer/adapters/base_adapter.rbindex 631c943e..231ef8a7 100644--- a/lib/blazer/adapters/base_adapter.rb+++ b/lib/blazer/adapters/base_adapter.rb@@ -8,7 +8,22 @@ module Blazerenddef run_statement(statement, comment)- # the one required method+ # required+ end++ def quoting+ # required, how to quote variables+ # :backslash_escape - single quote strings and convert ' to \' and \ to \\+ # :single_quote_escape - single quote strings and convert ' to ''+ # ->(value) { ... } - custom method+ end++ def parameter_binding+ # optional, but recommended when possible for security+ # if specified, quoting is only used for display+ # :positional - ?+ # :numeric - $1+ # ->(statement, values) { ... } - custom methodenddef tablesdiff --git a/lib/blazer/adapters/bigquery_adapter.rb b/lib/blazer/adapters/bigquery_adapter.rbindex 49db073c..b035da82 100644--- a/lib/blazer/adapters/bigquery_adapter.rb+++ b/lib/blazer/adapters/bigquery_adapter.rb@@ -1,13 +1,13 @@module Blazermodule Adaptersclass BigQueryAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)columns = []rows = []error = nilbegin- results = bigquery.query(statement)+ results = bigquery.query(statement, params: bind_params)# complete? was removed in google-cloud-bigquery 0.29.0# code is for backward compatibility@@ -19,6 +19,7 @@ module Blazerendrescue => eerror = e.message+ error = Blazer::VARIABLE_MESSAGE if error.include?("Syntax error: Unexpected \"?\"")end[columns, rows, error]@@ -42,6 +43,16 @@ module Blazer"SELECT * FROM `{table}` LIMIT 10"end+ # https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#string_and_bytes_literals+ def quoting+ :backslash_escape+ end++ # https://cloud.google.com/bigquery/docs/parameterized-queries+ def parameter_binding+ :positional+ end+privatedef bigquerydiff --git a/lib/blazer/adapters/cassandra_adapter.rb b/lib/blazer/adapters/cassandra_adapter.rbindex 3228c737..736c3c1c 100644--- a/lib/blazer/adapters/cassandra_adapter.rb+++ b/lib/blazer/adapters/cassandra_adapter.rb@@ -1,28 +1,29 @@module Blazermodule Adaptersclass CassandraAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)columns = []rows = []error = nilbegin- response = session.execute("#{statement} /*#{comment}*/")+ response = session.execute("#{statement} /*#{comment}*/", arguments: bind_params)rows = response.map { |r| r.values }columns = rows.any? ? response.first.keys : []rescue => eerror = e.message+ error = Blazer::VARIABLE_MESSAGE if error.include?("no viable alternative at input '?'")end[columns, rows, error]enddef tables- session.execute("SELECT table_name FROM system_schema.tables WHERE keyspace_name = '#{keyspace}'").map { |r| r["table_name"] }+ session.execute("SELECT table_name FROM system_schema.tables WHERE keyspace_name = #{data_source.quote(keyspace)}").map { |r| r["table_name"] }enddef schema- result = session.execute("SELECT keyspace_name, table_name, column_name, type, position FROM system_schema.columns WHERE keyspace_name = '#{keyspace}'")+ result = session.execute("SELECT keyspace_name, table_name, column_name, type, position FROM system_schema.columns WHERE keyspace_name = #{data_source.quote(keyspace)}")result.map(&:values).group_by { |r| [r[0], r[1]] }.map { |k, vs| {schema: k[0], table: k[1], columns: vs.sort_by { |v| v[2] }.map { |v| {name: v[2], data_type: v[3]} }} }end@@ -30,6 +31,16 @@ module Blazer"SELECT * FROM {table} LIMIT 10"end+ # https://docs.datastax.com/en/cql-oss/3.3/cql/cql_reference/escape_char_r.html+ def quoting+ :single_quote_escape+ end++ # https://docs.datastax.com/en/developer/nodejs-driver/3.0/features/parameterized-queries/+ def parameter_binding+ :positional+ end+privatedef clusterdiff --git a/lib/blazer/adapters/drill_adapter.rb b/lib/blazer/adapters/drill_adapter.rbindex 7db9c623..d5358d66 100644--- a/lib/blazer/adapters/drill_adapter.rb+++ b/lib/blazer/adapters/drill_adapter.rb@@ -18,6 +18,16 @@ module Blazer[columns, rows, error]end+ # https://drill.apache.org/docs/lexical-structure/#string+ def quoting+ :single_quote_escape+ end++ # https://issues.apache.org/jira/browse/DRILL-5079+ def parameter_binding+ # not supported+ end+privatedef drilldiff --git a/lib/blazer/adapters/druid_adapter.rb b/lib/blazer/adapters/druid_adapter.rbindex a99e6619..1eb3a4af 100644--- a/lib/blazer/adapters/druid_adapter.rb+++ b/lib/blazer/adapters/druid_adapter.rb@@ -3,7 +3,7 @@ module Blazerclass DruidAdapter < BaseAdapterTIMESTAMP_REGEX = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z\z/- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)require "json"require "net/http"require "uri"@@ -12,10 +12,28 @@ module Blazerrows = []error = nil+ params =+ bind_params.map do |v|+ type =+ case v+ when Integer+ "BIGINT"+ when Float+ "DOUBLE"+ when ActiveSupport::TimeWithZone+ v = (v.to_f * 1000).round+ "TIMESTAMP"+ else+ "VARCHAR"+ end+ {type: type, value: v}+ end+header = {"Content-Type" => "application/json", "Accept" => "application/json"}timeout = data_source.timeout ? data_source.timeout.to_i : 300data = {query: statement,+ parameters: params,context: {timeout: timeout * 1000}@@ -31,6 +49,8 @@ module Blazererror = response["errorMessage"] || "Unknown error: #{response.inspect}"if error.include?("timed out")error = Blazer::TIMEOUT_MESSAGE+ elsif error.include?("Encountered \"?\" at")+ error = Blazer::VARIABLE_MESSAGEendelsecolumns = (response.first || {}).keys@@ -66,6 +86,17 @@ module Blazerdef preview_statement"SELECT * FROM {table} LIMIT 10"end++ # https://druid.apache.org/docs/latest/querying/sql.html#identifiers-and-literals+ # docs only mention double quotes+ def quoting+ :single_quote_escape+ end++ # https://druid.apache.org/docs/latest/querying/sql.html#dynamic-parameters+ def parameter_binding+ :positional+ endendendenddiff --git a/lib/blazer/adapters/elasticsearch_adapter.rb b/lib/blazer/adapters/elasticsearch_adapter.rbindex 7e469d47..6488ae02 100644--- a/lib/blazer/adapters/elasticsearch_adapter.rb+++ b/lib/blazer/adapters/elasticsearch_adapter.rb@@ -1,13 +1,13 @@module Blazermodule Adaptersclass ElasticsearchAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)columns = []rows = []error = nilbegin- response = client.transport.perform_request("POST", endpoint, {}, {query: "#{statement} /*#{comment}*/"}).body+ response = client.transport.perform_request("POST", endpoint, {}, {query: "#{statement} /*#{comment}*/", params: bind_params}).bodycolumns = response["columns"].map { |v| v["name"] }# Elasticsearch does not differentiate between dates and timesdate_indexes = response["columns"].each_index.select { |i| ["date", "datetime"].include?(response["columns"][i]["type"]) }@@ -21,6 +21,7 @@ module Blazerendrescue => eerror = e.message+ error = Blazer::VARIABLE_MESSAGE if error.include?("mismatched input '?'")end[columns, rows, error]@@ -36,6 +37,16 @@ module Blazer"SELECT * FROM \"{table}\" LIMIT 10"end+ # https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-lexical-structure.html#sql-syntax-string-literals+ def quoting+ :single_quote_escape+ end++ # https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-rest-params.html+ def parameter_binding+ :positional+ end+protecteddef endpointdiff --git a/lib/blazer/adapters/hive_adapter.rb b/lib/blazer/adapters/hive_adapter.rbindex bd8cab16..4f2b4c81 100644--- a/lib/blazer/adapters/hive_adapter.rb+++ b/lib/blazer/adapters/hive_adapter.rb@@ -25,6 +25,16 @@ module Blazer"SELECT * FROM {table} LIMIT 10"end+ # https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-StringsstringStrings+ def quoting+ :backslash_escape+ end++ # has variable substitution, but sets for session+ # https://cwiki.apache.org/confluence/display/Hive/LanguageManual+VariableSubstitution+ def parameter_binding+ end+protecteddef clientdiff --git a/lib/blazer/adapters/ignite_adapter.rb b/lib/blazer/adapters/ignite_adapter.rbindex f345bf10..3b480f54 100644--- a/lib/blazer/adapters/ignite_adapter.rb+++ b/lib/blazer/adapters/ignite_adapter.rb@@ -1,13 +1,13 @@module Blazermodule Adaptersclass IgniteAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)columns = []rows = []error = nilbegin- result = client.query("#{statement} /*#{comment}*/", schema: default_schema, statement_type: :select, timeout: data_source.timeout)+ result = client.query("#{statement} /*#{comment}*/", bind_params, schema: default_schema, statement_type: :select, timeout: data_source.timeout)columns = result.any? ? result.first.keys : []rows = result.map(&:values)rescue => e@@ -37,6 +37,16 @@ module Blazer# result.rows.group_by { |r| [r[0], r[1]] }.map { |k, vs| {schema: k[0], table: k[1], columns: vs.sort_by { |v| v[2] }.map { |v| {name: v[2], data_type: v[3]} }} }.sort_by { |t| [t[:schema] == default_schema ? "" : t[:schema], t[:table]] }# end+ def quoting+ :single_quote_escape+ end++ # query arguments+ # https://ignite.apache.org/docs/latest/binary-client-protocol/sql-and-scan-queries#op_query_sql+ def parameter_binding+ :positional+ end+privatedef default_schemadiff --git a/lib/blazer/adapters/influxdb_adapter.rb b/lib/blazer/adapters/influxdb_adapter.rbindex ab59aa9b..1bab4f6d 100644--- a/lib/blazer/adapters/influxdb_adapter.rb+++ b/lib/blazer/adapters/influxdb_adapter.rb@@ -38,6 +38,15 @@ module Blazer"SELECT * FROM {table} LIMIT 10"end+ # https://docs.influxdata.com/influxdb/v1.8/query_language/spec/#strings+ def quoting+ :backslash_escape+ end++ def parameter_binding+ # not supported+ end+protecteddef clientdiff --git a/lib/blazer/adapters/mongodb_adapter.rb b/lib/blazer/adapters/mongodb_adapter.rbindex 350897f9..d4f188a0 100644--- a/lib/blazer/adapters/mongodb_adapter.rb+++ b/lib/blazer/adapters/mongodb_adapter.rb@@ -25,6 +25,10 @@ module Blazer"db.{table}.find().limit(10)"end+ def quoting+ :backslash_escape+ end+protecteddef clientdiff --git a/lib/blazer/adapters/neo4j_adapter.rb b/lib/blazer/adapters/neo4j_adapter.rbindex c2800dc6..35c79634 100644--- a/lib/blazer/adapters/neo4j_adapter.rb+++ b/lib/blazer/adapters/neo4j_adapter.rb@@ -1,13 +1,13 @@module Blazermodule Adaptersclass Neo4jAdapter < BaseAdapter- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params)columns = []rows = []error = nilbegin- result = session.query("#{statement} /*#{comment}*/")+ result = session.query("#{statement} /*#{comment}*/", bind_params)columns = result.columns.map(&:to_s)rows = []result.each do |row|@@ -19,6 +19,7 @@ module Blazerendrescue => eerror = e.message+ error = Blazer::VARIABLE_MESSAGE if error.include?("Invalid input '$'")end[columns, rows, error]@@ -33,6 +34,20 @@ module Blazer"MATCH (n:{table}) RETURN n LIMIT 10"end+ # https://neo4j.com/docs/cypher-manual/current/syntax/expressions/#cypher-expressions-string-literals+ def quoting+ :backslash_escape+ end++ def parameter_binding+ proc do |statement, variables|+ variables.each_key do |k|+ statement = statement.gsub("{#{k}}") { "$#{k} " }+ end+ [statement, variables]+ end+ end+protecteddef sessiondiff --git a/lib/blazer/adapters/opensearch_adapter.rb b/lib/blazer/adapters/opensearch_adapter.rbindex 5f6032e7..bca6181b 100644--- a/lib/blazer/adapters/opensearch_adapter.rb+++ b/lib/blazer/adapters/opensearch_adapter.rb@@ -38,6 +38,10 @@ module Blazer"SELECT * FROM `{table}` LIMIT 10"end+ def quoting+ # unknown+ end+protecteddef clientdiff --git a/lib/blazer/adapters/presto_adapter.rb b/lib/blazer/adapters/presto_adapter.rbindex 1ee0eddb..ab5bdc3c 100644--- a/lib/blazer/adapters/presto_adapter.rb+++ b/lib/blazer/adapters/presto_adapter.rb@@ -25,6 +25,15 @@ module Blazer"SELECT * FROM {table} LIMIT 10"end+ def quoting+ :single_quote_escape+ end++ # TODO support prepared statements - https://prestodb.io/docs/current/sql/prepare.html+ # feature request for variables - https://github.com/prestodb/presto/issues/5918+ def parameter_binding+ end+protecteddef clientdiff --git a/lib/blazer/adapters/salesforce_adapter.rb b/lib/blazer/adapters/salesforce_adapter.rbindex a0630ead..fdef968f 100644--- a/lib/blazer/adapters/salesforce_adapter.rb+++ b/lib/blazer/adapters/salesforce_adapter.rb@@ -35,6 +35,11 @@ module Blazer"SELECT Id FROM {table} LIMIT 10"end+ # https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_quotedstringescapes.htm+ def quoting+ :backslash_escape+ end+protecteddef clientdiff --git a/lib/blazer/adapters/snowflake_adapter.rb b/lib/blazer/adapters/snowflake_adapter.rbindex d14c41cf..46e6dea8 100644--- a/lib/blazer/adapters/snowflake_adapter.rb+++ b/lib/blazer/adapters/snowflake_adapter.rb@@ -68,6 +68,15 @@ module Blazerdef cancel(run_id)# todoend++ # https://docs.snowflake.com/en/sql-reference/data-types-text.html#escape-sequences+ def quoting+ :backslash_escape+ end++ def parameter_binding+ # TODO+ endendendenddiff --git a/lib/blazer/adapters/soda_adapter.rb b/lib/blazer/adapters/soda_adapter.rbindex 59b15a45..92050266 100644--- a/lib/blazer/adapters/soda_adapter.rb+++ b/lib/blazer/adapters/soda_adapter.rb@@ -95,6 +95,11 @@ module Blazerdef tables["all"]end++ # https://dev.socrata.com/docs/datatypes/text.html+ def quoting+ :single_quote_escape+ endendendenddiff --git a/lib/blazer/adapters/spark_adapter.rb b/lib/blazer/adapters/spark_adapter.rbindex 486982b0..0ac7b2fe 100644--- a/lib/blazer/adapters/spark_adapter.rb+++ b/lib/blazer/adapters/spark_adapter.rb@@ -4,6 +4,11 @@ module Blazerdef tablesclient.execute("SHOW TABLES").map { |r| r["tableName"] }end++ # https://spark.apache.org/docs/latest/sql-ref-literals.html+ def quoting+ :backslash_escape+ endendendenddiff --git a/lib/blazer/adapters/sql_adapter.rb b/lib/blazer/adapters/sql_adapter.rbindex 3df5a7ee..6908c695 100644--- a/lib/blazer/adapters/sql_adapter.rb+++ b/lib/blazer/adapters/sql_adapter.rb@@ -15,7 +15,7 @@ module Blazerendend- def run_statement(statement, comment)+ def run_statement(statement, comment, bind_params = [])columns = []rows = []error = nil@@ -24,7 +24,8 @@ module Blazerin_transaction doset_timeout(data_source.timeout) if data_source.timeout- result = select_all("#{statement} /*#{comment}*/")+ binds = bind_params.map { |v| ActiveRecord::Relation::QueryAttribute.new(nil, v, ActiveRecord::Type::Value.new) }+ result = connection_model.connection.select_all("#{statement} /*#{comment}*/", nil, binds)columns = result.columnsresult.rows.each do |untyped_row|rows << (result.column_types.empty? ? untyped_row : columns.each_with_index.map { |c, i| untyped_row[i] && result.column_types[c] ? result.column_types[c].send(:cast_value, untyped_row[i]) : untyped_row[i] })@@ -33,6 +34,7 @@ module Blazerrescue => eerror = e.message.sub(/.+ERROR: /, "")error = Blazer::TIMEOUT_MESSAGE if Blazer::TIMEOUT_ERRORS.any? { |e| error.include?(e) }+ error = Blazer::VARIABLE_MESSAGE if error.include?("syntax error at or near \"$") || error.include?("Incorrect syntax near '@") || error.include?("your MySQL server version for the right syntax to use near '?")reconnect if error.include?("PG::ConnectionBad")end@@ -156,7 +158,7 @@ module Blazer# WITH not an optimization fence in Postgres 12+statement = <<~SQLWITH query AS (- #{statement}+ {placeholder}),cohorts AS (SELECT user_id, MIN(#{cohort_column}) AS cohort_time FROM query@@ -183,6 +185,27 @@ module Blazerconnection_model.send(:sanitize_sql_array, params)end+ def quoting+ ->(value) { connection_model.connection.quote(value) }+ end++ # Redshift adapter silently ignores binds+ def parameter_binding+ if postgresql? || sqlite?+ :numeric+ elsif mysql? && connection_model.connection.prepared_statements?+ # Active Record silently ignores binds with MySQL when prepared statements are disabled+ :positional+ elsif sqlserver?+ proc do |statement, variables|+ variables.each_with_index do |(k, _), i|+ statement = statement.gsub("{#{k}}", "@#{i} ")+ end+ [statement, variables.values]+ end+ end+ end+protecteddef select_all(statement, params = [])@@ -207,6 +230,10 @@ module Blazer["MySQL", "Mysql2", "Mysql2Spatial"].include?(adapter_name)end+ def sqlite?+ ["SQLite"].include?(adapter_name)+ end+def sqlserver?["SQLServer", "tinytds", "mssql"].include?(adapter_name)enddiff --git a/lib/blazer/data_source.rb b/lib/blazer/data_source.rbindex b9de0b08..411a9486 100644--- a/lib/blazer/data_source.rb+++ b/lib/blazer/data_source.rb@@ -89,7 +89,19 @@ module BlazerBlazer.cache.delete(run_cache_key(run_id))end+ def sub_variables(statement, vars)+ statement = statement.dup+ vars.each do |var, value|+ # use block form to disable back-references+ statement.gsub!("{#{var}}") { quote(value) }+ end+ statement+ end+def run_statement(statement, options = {})+ statement = Statement.new(statement, self) if statement.is_a?(String)+ statement.bind unless statement.bind_statement+async = options[:async]result = nilif cache_mode != "off"@@ -118,7 +130,7 @@ module Blazerif options[:run_id]comment << ",run_id:#{options[:run_id]}"end- result = run_statement_helper(statement, comment, async ? options[:run_id] : nil)+ result = run_statement_helper(statement, comment, async ? options[:run_id] : nil, options)endresult@@ -133,13 +145,68 @@ module Blazerenddef statement_cache_key(statement)- cache_key(["statement", id, Digest::MD5.hexdigest(statement.to_s.gsub("\r\n", "\n"))])+ cache_key(["statement", id, Digest::MD5.hexdigest(statement.bind_statement.to_s.gsub("\r\n", "\n") + statement.bind_values.sort_by { |k, _| k }.to_json)])enddef run_cache_key(run_id)cache_key(["run", run_id])end+ def quote(value)+ if quoting == :backslash_escape || quoting == :single_quote_escape+ # only need to support types generated by process_vars+ if value.is_a?(Integer) || value.is_a?(Float)+ value.to_s+ elsif value.nil?+ "NULL"+ else+ value = value.to_formatted_s(:db) if value.is_a?(ActiveSupport::TimeWithZone)++ if quoting == :backslash_escape+ "'#{value.gsub("\\") { "\\\\" }.gsub("'") { "\\'" }}'"+ else+ "'#{value.gsub("'", "''")}'"+ end+ end+ elsif quoting.respond_to?(:call)+ quoting.call(value)+ elsif quoting.nil?+ raise Blazer::Error, "Quoting not specified"+ else+ raise Blazer::Error, "Unknown quoting"+ end+ end++ def bind_params(statement, variables)+ if parameter_binding == :positional+ locations = []+ variables.each do |k, v|+ i = 0+ while (idx = statement.index("{#{k}}", i))+ locations << [v, idx]+ i = idx + 1+ end+ end+ variables.each do |k, v|+ statement = statement.gsub("{#{k}}", "?")+ end+ [statement, locations.sort_by(&:last).map(&:first)]+ elsif parameter_binding == :numeric+ variables.each_with_index do |(k, v), i|+ # add trailing space if followed by digit+ # try to keep minimal to avoid fixing invalid queries like SELECT{var}+ statement = statement.gsub(/#{Regexp.escape("{#{k}}")}(\d)/, "$#{i + 1} \\1").gsub("{#{k}}", "$#{i + 1}")+ end+ [statement, variables.values]+ elsif parameter_binding.respond_to?(:call)+ parameter_binding.call(statement, variables)+ elsif parameter_binding.nil?+ [sub_variables(statement, variables), []]+ else+ raise Blazer::Error, "Unknown bind parameters"+ end+ end+protecteddef adapter_instance@@ -157,9 +224,22 @@ module Blazerendend- def run_statement_helper(statement, comment, run_id)+ def quoting+ @quoting ||= adapter_instance.quoting+ end++ def parameter_binding+ @parameter_binding ||= adapter_instance.parameter_binding+ end++ def run_statement_helper(statement, comment, run_id, options)start_time = Time.now- columns, rows, error = adapter_instance.run_statement(statement, comment)+ columns, rows, error =+ if adapter_instance.parameter_binding+ adapter_instance.run_statement(statement.bind_statement, comment, statement.bind_values)+ else+ adapter_instance.run_statement(statement.bind_statement, comment)+ endduration = Time.now - start_timecache_data = nil@@ -168,7 +248,7 @@ module Blazercache_data = Marshal.dump([columns, rows, error, cache ? Time.now : nil]) rescue nilend- if cache && cache_data && adapter_instance.cachable?(statement)+ if cache && cache_data && adapter_instance.cachable?(statement.bind_statement)Blazer.cache.write(statement_cache_key(statement), cache_data, expires_in: cache_expires_in.to_f * 60)enddiff --git a/lib/blazer/run_statement.rb b/lib/blazer/run_statement.rbindex 8ba3e163..bc5727f5 100644--- a/lib/blazer/run_statement.rb+++ b/lib/blazer/run_statement.rb@@ -1,12 +1,16 @@module Blazerclass RunStatement- def perform(data_source, statement, options = {})+ def perform(statement, options = {})query = options[:query]- Blazer.transform_statement.call(data_source, statement) if Blazer.transform_statement++ data_source = statement.data_source+ statement.bind# auditif Blazer.audit- audit = Blazer::Audit.new(statement: statement)+ audit_statement = statement.bind_statement+ audit_statement += "\n\n#{statement.bind_values.to_json}" if statement.bind_values.any?+ audit = Blazer::Audit.new(statement: audit_statement)audit.query = queryaudit.data_source = data_source.idaudit.user = options[:user]diff --git a/lib/blazer/run_statement_job.rb b/lib/blazer/run_statement_job.rbindex 8bc43921..7d7bfef4 100644--- a/lib/blazer/run_statement_job.rb+++ b/lib/blazer/run_statement_job.rb@@ -3,10 +3,12 @@ module Blazerself.queue_adapter = :asyncdef perform(data_source_id, statement, options)- data_source = Blazer.data_sources[data_source_id]+ statement = Blazer::Statement.new(statement, data_source_id)+ statement.values = options.delete(:values)+ data_source = statement.data_sourcebeginActiveRecord::Base.connection_pool.with_connection do- Blazer::RunStatement.new.perform(data_source, statement, options)+ Blazer::RunStatement.new.perform(statement, options)endrescue Exception => eBlazer::Result.new(data_source, [], [], "Unknown error", nil, false)diff --git a/lib/blazer/statement.rb b/lib/blazer/statement.rbnew file mode 100644index 00000000..994d733e--- /dev/null+++ b/lib/blazer/statement.rb@@ -0,0 +1,75 @@+module Blazer+ class Statement+ attr_reader :statement, :data_source, :bind_statement, :bind_values+ attr_accessor :values++ def initialize(statement, data_source = nil)+ @statement = statement+ @data_source = data_source.is_a?(String) ? Blazer.data_sources[data_source] : data_source+ @values = {}+ end++ def variables+ @variables ||= Blazer.extract_vars(statement)+ end++ def add_values(var_params)+ variables.each do |var|+ value = var_params[var].presence+ value = nil unless value.is_a?(String) # ignore arrays and hashes+ if value+ if ["start_time", "end_time"].include?(var)+ value = value.to_s.gsub(" ", "+") # fix for Quip bug+ end++ if var.end_with?("_at")+ begin+ value = Blazer.time_zone.parse(value)+ rescue+ # do nothing+ end+ end++ unless value.is_a?(ActiveSupport::TimeWithZone)+ if value =~ /\A\d+\z/+ value = value.to_i+ elsif value =~ /\A\d+\.\d+\z/+ value = value.to_f+ end+ end+ end+ value = Blazer.transform_variable.call(var, value) if Blazer.transform_variable+ @values[var] = value+ end+ end++ def cohort_analysis?+ /\/\*\s*cohort analysis\s*\*\//i.match?(statement)+ end++ def apply_cohort_analysis(period:, days:)+ @statement = data_source.cohort_analysis_statement(statement, period: period, days: days).sub("{placeholder}") { statement }+ end++ # should probably transform before cohort analysis+ # but keep previous order for now+ def transformed_statement+ statement = self.statement.dup+ Blazer.transform_statement.call(data_source, statement) if Blazer.transform_statement+ statement+ end++ def bind+ @bind_statement, @bind_values = data_source.bind_params(transformed_statement, values)+ end++ def display_statement+ data_source.sub_variables(transformed_statement, values)+ end++ def clear_cache+ bind if bind_statement.nil?+ data_source.clear_cache(self)+ end+ end+enddiff --git a/test/adapters/athena_test.rb b/test/adapters/athena_test.rbindex 684cd703..f8211071 100644--- a/test/adapters/athena_test.rb+++ b/test/adapters/athena_test.rb@@ -10,4 +10,64 @@ class AthenaTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ if engine_version > 1+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ else+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ if engine_version > 1+ assert_error "Exception parsing query", "SELECT 'world' AS {var}", var: "hello"+ else+ assert_error "mismatched input", "SELECT 'world' AS {var}", var: "hello"+ end+ end++ def test_quoted+ if engine_version > 1+ assert_error "Incorrect number of parameters: expected 0 but found 1", "SELECT '{var}' AS hello", var: "world"+ end+ end++ private++ def engine_version+ Blazer.data_sources[data_source].settings["engine_version"].to_i+ endenddiff --git a/test/adapters/bigquery_test.rb b/test/adapters/bigquery_test.rbindex 255982e0..ca3d16ea 100644--- a/test/adapters/bigquery_test.rb+++ b/test/adapters/bigquery_test.rb@@ -10,4 +10,50 @@ class BigqueryTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00 UTC"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ # TODO fix+ def test_nil+ assert_error "nil params are not supported, must assign optional type", "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ assert_bad_position "SELECT 'world' AS {var}", var: "hello"+ end++ # does not raise error for too many params+ def test_quoted+ assert_result [{"hello" => "?"}], "SELECT '{var}' AS hello", var: "world"+ endenddiff --git a/test/adapters/cassandra_test.rb b/test/adapters/cassandra_test.rbindex 00e84b35..7223cd6c 100644--- a/test/adapters/cassandra_test.rb+++ b/test/adapters/cassandra_test.rb@@ -17,12 +17,45 @@ class CassandraTest < ActionDispatch::IntegrationTestsession = cluster.connect("blazer_test")session.execute("DROP TABLE IF EXISTS items")- session.execute("CREATE TABLE items (id int, name text, PRIMARY KEY (id))")+ session.execute("CREATE TABLE items (id int, hello text, PRIMARY KEY (id))")+ session.execute("INSERT INTO items (id, hello) VALUES (1, 'world')")+ session.execute("INSERT INTO items (id, hello) VALUES (2, '''')")+ session.execute("INSERT INTO items (id, hello) VALUES (3, '\"')")+ session.execute("INSERT INTO items (id, hello) VALUES (4, '\\')")trueendend+ def test_tables+ assert_equal ["items"], tables+ end+def test_run- assert_result [{"count" => "0"}], "SELECT COUNT(*) FROM items"+ expected = [{"hello" => 'world'}]+ assert_result expected, "SELECT hello FROM items WHERE hello = 'world' ALLOW FILTERING"+ end++ def test_audit+ expected = "SELECT hello FROM items WHERE hello = ? ALLOW FILTERING\n\n[\"world\"]"+ assert_audit expected, "SELECT hello FROM items WHERE hello = {var} ALLOW FILTERING", var: "world"+ end++ def test_single_quote+ expected = [{"hello" => "'"}]+ assert_result expected, "SELECT hello FROM items WHERE hello = {var} ALLOW FILTERING", var: "'"+ end++ def test_double_quote+ expected = [{"hello" => '"'}]+ assert_result expected, "SELECT hello FROM items WHERE hello = {var} ALLOW FILTERING", var: '"'+ end++ def test_backslash+ expected = [{"hello" => "\\"}]+ assert_result expected, "SELECT hello FROM items WHERE hello = {var} ALLOW FILTERING", var: "\\"+ end++ def test_bad_position+ assert_bad_position "SELECT hello FROM items WHERE hello {var} 'world' ALLOW FILTERING", var: "="endenddiff --git a/test/adapters/drill_test.rb b/test/adapters/drill_test.rbindex 2132a8e4..bbbd5e5e 100644--- a/test/adapters/drill_test.rb+++ b/test/adapters/drill_test.rb@@ -10,4 +10,20 @@ class DrillTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ endenddiff --git a/test/adapters/druid_test.rb b/test/adapters/druid_test.rbindex c5471b5f..c549bc9b 100644--- a/test/adapters/druid_test.rb+++ b/test/adapters/druid_test.rb@@ -10,4 +10,45 @@ class DruidTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00 UTC"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ # TODO fix+ def test_nil+ # assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ assert_bad_position "SELECT hello AS {var}", var: "world"+ endenddiff --git a/test/adapters/elasticsearch_test.rb b/test/adapters/elasticsearch_test.rbindex b48fee3b..d30c70ce 100644--- a/test/adapters/elasticsearch_test.rb+++ b/test/adapters/elasticsearch_test.rb@@ -10,4 +10,44 @@ class ElasticsearchTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01T08:00:00.000Z"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ assert_bad_position "SELECT 'world' AS {var}", var: "hello"+ endenddiff --git a/test/adapters/hive_test.rb b/test/adapters/hive_test.rbindex 34b967f6..0e43aa70 100644--- a/test/adapters/hive_test.rb+++ b/test/adapters/hive_test.rb@@ -1,5 +1,7 @@require_relative "../test_helper"+# bin/beeline -u jdbc:hive2://localhost:10000 -e 'CREATE DATABASE blazer_test;'+class HiveTest < ActionDispatch::IntegrationTestinclude AdapterTest@@ -10,4 +12,40 @@ class HiveTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ endenddiff --git a/test/adapters/ignite_test.rb b/test/adapters/ignite_test.rbindex 8ecde393..bb1eb59d 100644--- a/test/adapters/ignite_test.rb+++ b/test/adapters/ignite_test.rb@@ -10,4 +10,44 @@ class IgniteTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"HELLO" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"HELLO" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"HELLO" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"HELLO" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"HELLO" => "2022-01-01 08:00:00 UTC"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"HELLO" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"HELLO" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"HELLO" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"HELLO" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ assert_error "Syntax error in SQL statement", "SELECT 'world' AS {var}", var: "hello"+ endenddiff --git a/test/adapters/influxdb_test.rb b/test/adapters/influxdb_test.rbindex d620e0dd..72b1103b 100644--- a/test/adapters/influxdb_test.rb+++ b/test/adapters/influxdb_test.rb@@ -12,12 +12,37 @@ class InfluxdbTest < ActionDispatch::IntegrationTestclient = InfluxDB::Client.new(url: "http://localhost:8086/blazer_test")client.delete_series("items")client.write_point("items", {values: {value: 1}, tags: {hello: "world"}, timestamp: 0})+ client.write_point("items", {values: {value: 1}, tags: {hello: "'"}, timestamp: 0})+ client.write_point("items", {values: {value: 1}, tags: {hello: '"'}, timestamp: 0})+ # InfluxDB does not like trailing backslashes+ # https://github.com/influxdata/influxdb/issues/5231+ # https://github.com/influxdata/influxdb-ruby/issues/225+ client.write_point("items", {values: {value: 1}, tags: {hello: "\\a"}, timestamp: 0})trueendenddef test_run- expected = [{"time" => "1970-01-01 00:00:00 UTC", "count_value" => "1"}]- assert_result expected, "SELECT COUNT(*) FROM items WHERE hello = 'world'"+ expected = [{"time" => "1970-01-01 00:00:00 UTC", "hello" => "world", "value" => "1"}]+ assert_result expected, "SELECT * FROM items WHERE hello = 'world'"+ end++ def test_audit+ assert_audit "SELECT * FROM items WHERE hello = 'world'", "SELECT * FROM items WHERE hello = {var}", var: "world"+ end++ def test_single_quote+ expected = [{"time" => "1970-01-01 00:00:00 UTC", "hello" => "'", "value" => "1"}]+ assert_result expected, "SELECT * FROM items WHERE hello = {var}", var: "'"+ end++ def test_double_quote+ expected = [{"time" => "1970-01-01 00:00:00 UTC", "hello" => '"', "value" => "1"}]+ assert_result expected, "SELECT * FROM items WHERE hello = {var}", var: '"'+ end++ def test_backslash+ expected = [{"time" => "1970-01-01 00:00:00 UTC", "hello" => "\\a", "value" => "1"}]+ assert_result expected, "SELECT * FROM items WHERE hello = {var}", var: "\\a"endenddiff --git a/test/adapters/mongodb_test.rb b/test/adapters/mongodb_test.rbindex 154a1136..f8040b13 100644--- a/test/adapters/mongodb_test.rb+++ b/test/adapters/mongodb_test.rb@@ -12,11 +12,30 @@ class MongodbTest < ActionDispatch::IntegrationTestclient = Mongo::Client.new(["127.0.0.1:27017"], database: "blazer_test")client[:items].dropclient[:items].insert_one({hello: "world"})+ client[:items].insert_one({hello: "'"})+ client[:items].insert_one({hello: '"'})+ client[:items].insert_one({hello: "\\"})trueendenddef test_run- assert_result [{"hello" => "world"}], "db.items.find({}, {hello: 1, _id: 0})"+ assert_result [{"hello" => "world"}], "db.items.find({hello: 'world'}, {hello: 1, _id: 0})"+ end++ def test_audit+ assert_audit "db.items.find({hello: 'world'}, {hello: 1, _id: 0})", "db.items.find({hello: {var}}, {hello: 1, _id: 0})", var: "world"+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "db.items.find({hello: {var}}, {hello: 1, _id: 0})", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "db.items.find({hello: {var}}, {hello: 1, _id: 0})", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "db.items.find({hello: {var}}, {hello: 1, _id: 0})", var: "\\"endenddiff --git a/test/adapters/mysql_test.rb b/test/adapters/mysql_test.rbindex 77526267..99e661c2 100644--- a/test/adapters/mysql_test.rb+++ b/test/adapters/mysql_test.rb@@ -10,4 +10,86 @@ class MysqlTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ if prepared_statements?+ assert_audit "SELECT ? AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ else+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_multiple_variables+ assert_result [{"c1" => "one", "c2" => "two", "c3" => "one"}], "SELECT {var} AS c1, {var2} AS c2, {var} AS c3", var: "one", var2: "two"+ end++ def test_bad_position+ if prepared_statements?+ assert_bad_position "SELECT 'world' AS {var}", var: "hello"+ else+ assert_result [{"hello"=>"world"}], "SELECT 'world' AS {var}", var: "hello"+ end+ end++ def test_bad_position_before+ if prepared_statements?+ assert_result [{"?" => "world"}], "SELECT{var}", var: "world"+ else+ assert_result [{"world" => "world"}], "SELECT{var}", var: "world"+ end+ end++ def test_bad_position_after+ if prepared_statements?+ assert_bad_position "SELECT {var}456", var: "world"+ else+ assert_error "You have an error in your SQL syntax", "SELECT {var}456", var: "world"+ end+ end++ def test_quoted+ if prepared_statements?+ assert_error "Bind parameter count (0) doesn't match number of arguments (1)", "SELECT '{var}' AS hello", var: "world"+ else+ assert_error "You have an error in your SQL syntax", "SELECT '{var}' AS hello", var: "world"+ end+ end++ private++ def prepared_statements?+ Blazer.data_sources[data_source].settings["url"].include?("prepared_statements=true")+ endenddiff --git a/test/adapters/neo4j_test.rb b/test/adapters/neo4j_test.rbindex c079a372..0a1e14af 100644--- a/test/adapters/neo4j_test.rb+++ b/test/adapters/neo4j_test.rb@@ -10,4 +10,44 @@ class Neo4jTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "OPTIONAL MATCH () RETURN 'world' AS `hello`"end++ def test_audit+ assert_audit "OPTIONAL MATCH () RETURN $var AS `hello`\n\n{\"var\":\"world\"}", "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01T08:00:00.000Z"}], "OPTIONAL MATCH () RETURN {created_at} AS `hello`", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "OPTIONAL MATCH () RETURN {var} AS `hello`", var: "\\"+ end++ def test_bad_position+ assert_bad_position "OPTIONAL MATCH () RETURN 'world' AS {var}", var: "hello"+ endenddiff --git a/test/adapters/opensearch_test.rb b/test/adapters/opensearch_test.rbindex 79ee678b..b2d06b13 100644--- a/test/adapters/opensearch_test.rb+++ b/test/adapters/opensearch_test.rb@@ -10,4 +10,16 @@ class OpensearchTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"'world'" => "world"}], "SELECT 'world' AS hello"end++ def test_single_quote+ assert_error "Quoting not specified", "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_error "Quoting not specified", "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_error "Quoting not specified", "SELECT {var} AS hello", var: "\\"+ endenddiff --git a/test/adapters/postgresql_test.rb b/test/adapters/postgresql_test.rbindex 76600f88..6828ce58 100644--- a/test/adapters/postgresql_test.rb+++ b/test/adapters/postgresql_test.rb@@ -10,4 +10,61 @@ class PostgresqlTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT $1 AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_multiple_variables+ assert_result [{"c1" => "one", "c2" => "two", "c3" => "one"}], "SELECT {var} AS c1, {var2} AS c2, {var} AS c3", var: "one", var2: "two"+ end++ def test_bad_position+ assert_bad_position "SELECT 'world' AS {var}", var: "hello"+ end++ def test_bad_position_before+ assert_error "syntax error at or near \"SELECT$1\"", "SELECT{var}", var: "world"+ end++ def test_bad_position_after+ assert_error "syntax error at or near \"456\"\nLINE 1: SELECT $1 456", "SELECT {var}456", var: "world"+ assert_equal "SELECT $1 456\n\n[\"world\"]", Blazer::Audit.last.statement+ end++ def test_quoted+ assert_error "could not determine data type of parameter $1", "SELECT '{var}' AS hello", var: "world"+ endenddiff --git a/test/adapters/presto_test.rb b/test/adapters/presto_test.rbindex 33e93a35..518d385d 100644--- a/test/adapters/presto_test.rb+++ b/test/adapters/presto_test.rb@@ -14,4 +14,20 @@ class PrestoTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ endenddiff --git a/test/adapters/redshift_test.rb b/test/adapters/redshift_test.rbindex 77dc6512..5e36df48 100644--- a/test/adapters/redshift_test.rb+++ b/test/adapters/redshift_test.rb@@ -6,4 +6,52 @@ class RedshiftTest < ActionDispatch::IntegrationTestdef data_source"redshift"end++ def test_run+ assert_result [{"hello" => "world"}], "SELECT 'world' AS hello"+ end++ def test_audit+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_multiple_variables+ assert_result [{"c1" => "one", "c2" => "two", "c3" => "one"}], "SELECT {var} AS c1, {var2} AS c2, {var} AS c3", var: "one", var2: "two"+ end++ def test_bad_position+ assert_error "syntax error at or near", "SELECT 'world' AS {var}", var: "hello"+ endenddiff --git a/test/adapters/salesforce_test.rb b/test/adapters/salesforce_test.rbindex 8c48d781..52ee193f 100644--- a/test/adapters/salesforce_test.rb+++ b/test/adapters/salesforce_test.rb@@ -1,9 +1,35 @@require_relative "../test_helper"+# https://stackoverflow.com/questions/12794302/salesforce-authentication-failing/29112224#29112224+# create accounts named world, ', ", and \++# ENV["SALESFORCE_USERNAME"] = "username"+# ENV["SALESFORCE_PASSWORD"] = "password"+# ENV["SALESFORCE_SECURITY_TOKEN"] = "security token"+# ENV["SALESFORCE_CLIENT_ID"] = "client id"+# ENV["SALESFORCE_CLIENT_SECRET"] = "client secret"+# ENV["SALESFORCE_API_VERSION"] = "41.0"+class SalesforceTest < ActionDispatch::IntegrationTestinclude AdapterTestdef data_source"salesforce"end++ def test_run+ assert_result [{"Name" => "world"}], "SELECT Name FROM Account WHERE Name = 'world'"+ end++ def test_single_quote+ assert_result [{"Name" => "'"}], "SELECT Name FROM Account WHERE Name = {var}", var: "'"+ end++ def test_double_quote+ assert_result [{"Name" => '"'}], "SELECT Name FROM Account WHERE Name = {var}", var: '"'+ end++ def test_backslash+ assert_result [{"Name" => "\\"}], "SELECT Name FROM Account WHERE Name = {var}", var: "\\"+ endenddiff --git a/test/adapters/soda_test.rb b/test/adapters/soda_test.rbindex 483d0097..16bbb358 100644--- a/test/adapters/soda_test.rb+++ b/test/adapters/soda_test.rb@@ -14,4 +14,16 @@ class SodaTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello LIMIT 1"end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello LIMIT 1", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello LIMIT 1", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello LIMIT 1", var: "\\"+ endenddiff --git a/test/adapters/spark_test.rb b/test/adapters/spark_test.rbindex 662ae0a0..1ba86c7e 100644--- a/test/adapters/spark_test.rb+++ b/test/adapters/spark_test.rb@@ -12,4 +12,40 @@ class SparkTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT 'world' AS hello", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ endenddiff --git a/test/adapters/sqlite_test.rb b/test/adapters/sqlite_test.rbindex 1f71f755..dbb1d331 100644--- a/test/adapters/sqlite_test.rb+++ b/test/adapters/sqlite_test.rb@@ -10,4 +10,44 @@ class SqliteTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT $1 AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ def test_float+ assert_result [{"hello" => "1.5"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "2022-01-01 08:00:00"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_multiple_variables+ assert_result [{"c1" => "one", "c2" => "two", "c3" => "one"}], "SELECT {var} AS c1, {var2} AS c2, {var} AS c3", var: "one", var2: "two"+ endenddiff --git a/test/adapters/sqlserver_test.rb b/test/adapters/sqlserver_test.rbindex d490a92d..7f1d389b 100644--- a/test/adapters/sqlserver_test.rb+++ b/test/adapters/sqlserver_test.rb@@ -14,4 +14,50 @@ class SqlserverTest < ActionDispatch::IntegrationTestdef test_runassert_result [{"hello" => "world"}], "SELECT 'world' AS hello"end++ def test_audit+ assert_audit "SELECT @0 AS hello\n\n[\"world\"]", "SELECT {var} AS hello", var: "world"+ end++ def test_string+ assert_result [{"hello" => "world"}], "SELECT {var} AS hello", var: "world"+ end++ def test_integer+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1"+ end++ # https://github.com/rails-sqlserver/activerecord-sqlserver-adapter/issues/643+ # should be 1.5+ def test_float+ assert_result [{"hello" => "1"}], "SELECT {var} AS hello", var: "1.5"+ end++ def test_time+ assert_result [{"hello" => "01-01-2022 08:00:00.0"}], "SELECT {created_at} AS hello", created_at: "2022-01-01 08:00:00"+ end++ def test_nil+ assert_result [{"hello" => nil}], "SELECT {var} AS hello", var: ""+ end++ def test_single_quote+ assert_result [{"hello" => "'"}], "SELECT {var} AS hello", var: "'"+ end++ def test_double_quote+ assert_result [{"hello" => '"'}], "SELECT {var} AS hello", var: '"'+ end++ def test_backslash+ assert_result [{"hello" => "\\"}], "SELECT {var} AS hello", var: "\\"+ end++ def test_bad_position+ assert_bad_position "SELECT 'world' AS {var}", var: "hello"+ end++ def test_quoted+ assert_result [{"hello"=>"@0 "}], "SELECT '{var}' AS hello", var: "world"+ endenddiff --git a/test/internal/config/blazer.yml b/test/internal/config/blazer.ymlindex da4f6203..07c7b09d 100644--- a/test/internal/config/blazer.yml+++ b/test/internal/config/blazer.yml@@ -75,7 +75,7 @@ data_sources:url: mongodb://localhost:27017/blazer_testmysql:- url: mysql2://localhost:3306/blazer_test+ url: mysql2://localhost:3306/blazer_test # ?prepared_statements=trueneo4j:adapter: neo4jdiff --git a/test/support/adapter_test.rb b/test/support/adapter_test.rbindex 03667bea..c3251ad1 100644--- a/test/support/adapter_test.rb+++ b/test/support/adapter_test.rb@@ -12,12 +12,28 @@ module AdapterTestJSON.parse(response.body)end- def assert_result(expected, statement)- assert_equal expected, run_statement(statement)+ def assert_result(expected, statement, **params)+ assert_equal expected, run_statement(statement, **params)end- def run_statement(statement)- run_query statement, data_source: data_source, format: "csv"+ def assert_audit(expected, statement, **params)+ run_statement(statement, **params)+ assert_equal expected, Blazer::Audit.last.statement+ end++ def assert_error(message, statement, **params)+ error = assert_raises(Blazer::Error) do+ run_statement(statement, **params)+ end+ assert_match message, error.message+ end++ def assert_bad_position(statement, **params)+ assert_error "Variable cannot be used in this position", statement, **params+ end++ def run_statement(statement, **params)+ run_query statement, **params, data_source: data_source, format: "csv"CSV.parse(response.body, headers: true).map(&:to_h)endend