歡迎來到Linux教程網
Linux教程網
Linux教程網
Linux教程網
Linux教程網 >> Linux編程 >> Linux編程 >> 用newLISP讀取Hive的元數據

用newLISP讀取Hive的元數據

日期:2017/3/1 9:43:45   编辑:Linux編程

思想是通過調用hive -e命令,然後解析返回的結果。

下面的hive.lsp文件是一個例子,提供了讀取本地hive數據庫名稱,表名,表結構和根據hive表的創建語句生成對應的MySQL的創建語句。

最新代碼參考我在gitlab.com上的開源項目:https://gitlab.com/newlisp/armory.git

;; hive.lsp

;; @module hive
;; @description hive module provides some features for manuplating local hive database
;; @note must make sure the hive env are correct
;; @location hive.lsp
;; @version 0.1.0
;; @author Dean Chen
;; @example
;; (load "/opt/newlisp_util/env.lsp")
;; (HIVE:show-dbs)
;; (HIVE:show-tables db-name)
;; (HIVE:show-create-table-statment db-name table-name)

(context 'HIVE)

;; @syntax (HIVE:show-dbs)
(define (show-dbs)
(exec (format "hive -e 'show databases' 2>&1")))

;; @syntax (HIVE:show-tables db-name)
(define (show-tables db-name)
(exec (format "hive -e 'use %s; show tables' 2>&1" db-name)))

;; @syntax (HIVE:describe-table db-name table-name)
(define (describe-table db-name table-name)
(exec (format "hive -e 'use %s; describe %s' 2>&1" db-name table-name)))

;; @syntax (HIVE:show-create-table db-name table-name)
(define (show-create-table db-name table-name)
(exec (format "hive -e 'use %s; show create table %s' 2>&1" db-name table-name)))

;; @syntax (HIVE:gen-mysql-create-table hive-db-name hive-table-name)
;; @note this method use list index to locate CREATE EXTERNAL statement
;; it could be improved later for getting more compatibilty
(define (gen-mysql-create-table db-name table-name)
(set 's-list (show-create-table db-name table-name))
(set 'header (replace "CREATE EXTERNAL" (s-list 6) "CREATE"))
(set 'mysql-create-table-statement "")
(set 'mysql-create-table-statement (append mysql-create-table-statement header))
(set 'body (slice s-list 7))
(catch (dolist (field body)
(if (regex "PARTITIONED BY" field)
(begin
(set 'sub-length (- (length mysql-create-table-statement) 1))
(set 'mysql-create-table-statement (slice mysql-create-table-statement 0 sub-length))
(throw mysql-create-table-statement))
(begin
(set 'column-list (parse field ","))
(dolist (column column-list)
(set 'column-name (parse (trim column) " "))
(if column-name
(begin
(set 'mysql-create-table-statement (append mysql-create-table-statement (column-name 0)))
(set 'mysql-create-table-statement (append mysql-create-table-statement " "))
(set 'mysql-create-table-statement (append mysql-create-table-statement (replace "string" (column-name 1) "varchar(1024)")))
(set 'mysql-create-table-statement (append mysql-create-table-statement ","))
)
)))))))

newLISP 的詳細介紹:請點這裡
newLISP 的下載地址:請點這裡

為Emacs配置newLISP開發環境 http://www.linuxidc.com/Linux/2013-01/78463.htm

newLISP做GitLab系統備份 http://www.linuxidc.com/Linux/2013-01/78464.htm

newLISP 遍歷目錄樹,清理編譯目錄 http://www.linuxidc.com/Linux/2013-08/88954.htm

Copyright © Linux教程網 All Rights Reserved