hdfs (2) 썸네일형 리스트형 [python] get hdfs path info import requests import pandas as pd def getPathInfo(hdfsPath): try: s = requests.session() userName = "hive" #getpass.getuser() operation = "LISTSTATUS" httpFsUrl = "http://hostip:14000" req = "{0}/webhdfs/v1{1}?user.name={2}&op={3}".format(httpFsUrl,hdfsPath,userName,operation) response = s.get(req,headers={'Connection':'close'}) result = response.json()["FileStatuses"]["FileStatus"] dfTmp = pd.. [hdfs] webhdfs httpfs api (file put/get) hdfs command $ hadoop fs -put /localfile /hdfsdir/hdfsfilename $ hadoop fs -get /hdfsfilename /localdir/localfile webhdfs api command curl -i "http://host:14000/webhdfs/v1?op=gethomedirectory&user.name=hive" curl -i "http://host:14000/webhdfs/v1/user/hive/test.txt?op=open&user.name=hive" -> return file contents curl -i -X GET "http://host:14000/webhdfs/v1/user/hive?op=LISTSTATUS&user.name=hive" .. 이전 1 다음