HEX

File: //opt/alt/python37/lib/python3.7/site-packages/xray/apiclient/__pycache__/api_client.cpython-37.pyc
B

�*Udif�@sxdZddlZddlZddlZddlZddlZddlZddlm	Z	ddl
mZmZddl
mZmZddlmZddlmZddlmZd	d
lmZd	dlmZmZmZd	dlmZd	d
lm Z d	dl!m"Z"d	dl#m$Z$m%Z%d	dl&m'Z'Gdd�d�Z(Gdd�d�Z)Gdd�de(e)�Z*Gdd�de(�Z+Gdd�de(e)�Z,Gdd�de(�Z-Gdd�de(�Z.Gdd �d e(�Z/Gd!d"�d"e(�Z0dS)#zA
This module contains class implementing MongoDB API interaction
�N)�partial)�List�Any)�Session�Response)�HTTPAdapter)�RequestException)�Retry�)�	supported)�
api_server�proto�adviser_api_server)�XRayAPIError)�open_local_storage)�Task)�get_xray_exec_user�user_mode_verification)�read_jwt_tokenc@s�eZdZdZdd�eed�dd�Zdd�Zd	d
�Zeed�dd
��Z	ee
d�dd��Zed�dd�Zd)e
eed�dd�Zd*e
ee
d�dd�Zd+ee
eee
d�dd�Zd,eeed�dd�Zd-ee
d�d d!�Zd.eeed"�d#d$�Zd/ee
d�d%d&�Zd'd(�ZdS)0�Clientz
    Base client class
    �unavailable)�tracing_task_id)�	system_idrcCsx||_||_t�d�|_tdtddg�tdddg�dd�}t|d	�}t�|_	|j	�
t�d
�|�t|j	j
dd�|j	_
dS)
N�
api_client��GET�POSTi�i�i�)�total�method_whitelist�status_forcelist�backoff_factor)�max_retriesz://�
)�timeout)r�task_id�logging�	getLogger�loggerr	�	frozensetrr�session�mountr
r�request)�selfrrZ
retry_conf�adapter�r.�J/opt/alt/python37/lib/python3.7/site-packages/xray/apiclient/api_client.py�__init__*s

zClient.__init__cCs|jj�d|j�d|j��S)Nz::z::tracing_task_id=)�	__class__�__name__�
main_endpointr$)r,r.r.r/�__repr__9szClient.__repr__cCs|jj�d|j��S)Nz::)r1r2r$)r,r.r.r/�__str__<szClient.__str__)�returncCstd��dS)z
        Base endpoint
        z$instances are to set their endpoint!N)�NotImplementedError)r,r.r.r/r3?szClient.main_endpointcCs
dt�iS)zL
        Updated request headers
        :return: dict with headers
        zX-Auth)r)r,r.r.r/�_headersFszClient._headerscKs$d|ji}|r|�|�tj�|�S)ze
        Construct query string
        :return: string including system_id and given kwargs
        r)r�update�urllib�parse�	urlencode)r,�kwargs�initialr.r.r/�
_query_stringPs

zClient._query_stringT)�response�with_status_checkr6cCs�|js:|j�d|j�td|j�d|j��d|jid��n|j�d|j|j|j�|jdkrf|�	�}ndd	i}|r�|dd	kr�td
|����|S)z�
        Perform preprocessing checks of received Response:
        - is it OK, e.g. 200 OK
        - was it successful, e.g. status == ok
        and extract JSON from the Response object
        :return: full JSON representation of response
        zServer responded: %sz!Unable to connect to server with �:Z	resp_data)�extraz[%s:%s] Response received %s���status�okz Received unsuccessful response: )
rFr'�warning�textr�status_code�reason�info�url�json)r,r@rA�resultr.r.r/�_preprocess_responseZs	

zClient._preprocess_responsecCs^|j||d�}y4|d}|j�d|j|�|dkr@td|����|Stk
rXt�SXdS)z�
        Check received response
        :param response: a requests.Response object
        :return: 'result' dict from JSON representation of response
        )rArNz[%s] Received response data %sNzReceived empty result: )rOr'rKrIr�KeyError�dict)r,r@rArN�datar.r.r/�_process_responsews
zClient._process_responseN)�endpoint�payload�log_datarAr6c
Cs�|j�d|�|r$|r$|j�d|�y6|dkrB|jj||jd�}n|jj|||jdd�}Wn4tk
r�}z|�||dd�|�Wdd}~XYnX|j||d	�S)
a.
        Perform POST request to given endpoint.
        Add payload as JSON if given
        :param endpoint: target URL
        :param payload: dict with date to POST
        :param log_data: whether to log POST data or not
        :return: 'result' dict from JSON representation of response
        zSending POST request to %szData attached to POST: %sN)�headers�<)rMrWr#zPOST failedz'Failed to POST data to X-Ray API server)rA)r'rKr)�postr8r�_give_xray_exceptionrS)r,rTrUrVrA�resp�er.r.r/�_post�s

zClient._post)rTrVrAc
Csh|j�d|�y|jj||jd�}Wn4tk
rX}z|�||dd�|�Wdd}~XYnX|j||d�S)NzSending DELETE request to %s)rWz
DELETE failedz)Failed to DE:ETE data to X-Ray API server)rA)r'rKr)�deleter8rrZrS)r,rTrVrAr[r\r.r.r/�_delete�s
zClient._delete)rTr6c
Csz|dkr|j�d|����}|j�d|�y|jj||jd�}Wn4tk
rt}z|�||dd�|�Wdd}~XYnX|S)z�
        GET request to endpoint or to main endpoint if no endpoint given
        :param endpoint: target URL
        :return: a requests Response object
        N�?zSending GET request to %s)rWz
GET failedz(Failed to GET data from X-Ray API server)	r3r?r'rKr)�getr8rrZ)r,rTr[r\r.r.r/�_raw_get�s
zClient._raw_get)rTrAr6cCs|�|�}|j||d�S)z�
        GET request to endpoint or to main endpoint if no endpoint given
        :param endpoint: target URL
        :return: full dict from JSON representation of response without any
                 processing
        )rA)rbrO)r,rTrAr[r.r.r/�	_get_full�s
zClient._get_fullcCs|�|�}|�|�S)z�
        GET request to endpoint or to main endpoint if no endpoint given
        :param endpoint: target URL
        :return: 'result' dict from JSON representation of response
        )rbrS)r,rTr[r.r.r/�_get�s
zClient._getc	CsZ|jjd||d|id�y|jdj}Wnttfk
rD|}YnXt|�d|�d��S)z�
        Process received exception
        :param exc: original exception
        :param api_endpoint: requested endpoint
        :param log_message: text for logging the error
        :param exc_message: text for internal exception
        z
%s with %srT)rCrz: z. Please, try again later)r'�error�argsrJ�
IndexError�AttributeErrorr)r,�excZapi_endpoint�log_message�exc_message�exc_infor.r.r/rZ�s	
zClient._give_xray_exception)T)T)NTT)TT)N)NT)N)r2�
__module__�__qualname__�__doc__�strr0r4r5�propertyr3rQr8r?r�boolrrOrSr]r_rbrcrdrZr.r.r.r/r%s(		rc@s2eZdZdZeed�dd��Zeed�dd�Z	dS)	�	TaskMixinz1
    A mixin class with Task related methods
    )r6cCsdS)z(
        Limit processed fields
        )rLrE�	client_ip�
tracing_by�
tracing_count�	starttime�ini_location�
initial_count�
request_count�	auto_task�userr.)r,r.r.r/�task_fields�szTaskMixin.task_fields)�	dict_viewr6cs,�fdd�|��D�}|d|d<tf|�S)z@
        Turn dictionary structure into valid Task type
        cs i|]\}}|�jkr||�qSr.)r})�.0�k�v)r,r.r/�
<dictcomp>�sz#TaskMixin._task.<locals>.<dictcomp>rr$)�itemsr)r,r~�	task_viewr.)r,r/�_task�szTaskMixin._taskN)
r2rmrnrorq�tupler}rQrr�r.r.r.r/rs�srscsNeZdZdZeed�dd��Zed��fdd�Zee	d��fdd�Z
�ZS)	�TasksClientz!
    'tasks' endpoint client
    )r6cCst�dt�d�S)z&
        Base endpoint: tasks
        z://z/api/xray/tasks)r
r)r,r.r.r/r3szTasksClient.main_endpointcst�jdd�S)z�
        Construct query string.
        Aimed to get auto tasks only
        :return: string including system_id and type=auto
        �auto)�type)�superr?)r,r=)r1r.r/r?
szTasksClient._query_stringcst���}�fdd�|D�S)z#
        Get list of Tasks
        csg|]}��|��qSr.)r�)r�item)r,r.r/�
<listcomp>sz)TasksClient.get_tasks.<locals>.<listcomp>)r�rd)r,rR)r1)r,r/�	get_taskss
zTasksClient.get_tasks)r2rmrnrorqrpr3r?rrr��
__classcell__r.r.)r1r/r�s
r�cs�eZdZdZdd�eed��fdd�Zdd�Zed	d
�dd��Ze	d
d�dd�Z
e	d
�dd�Zeed
�dd��Ze
d
�dd�Ze
ed�dd�Zed
d�dd�Zed
d�dd�Zd
d
�d d!�Zd
d
�d"d#�Z�ZS)$�	DBMClientzD
    Client class using local dbm storage instead of remote API
    r)r)rrcs$t�j||d�d|_|��|_dS)N)rr)r�r0�task_object�_db_open�_db)r,rr)r1r.r/r0"szDBMClient.__init__cCs|j��dS)N)r��close)r,r.r.r/�__del__(szDBMClient.__del__zgdbm object)r6cCst�dd�S)zC
        Open dbm DB
        :return: corresponding object
        z/root/local_mongo�c)�dbm�openr.r.r.r/r�+szDBMClient._db_openN)�	post_datar6cCst�|�|j|j<dS)z3
        Update a DBM task with given data
        N)rM�dumpsr�r$)r,r�r.r.r/r]3szDBMClient._postc
CsNyt�|j|j���Sttjfk
rH}ztd�|�Wdd}~XYnXdS)z:
        Get saved DBM data
        :return: dict
        zFailed to load taskN)rM�loadsr�r$�decoderP�JSONDecodeErrorr)r,r\r.r.r/rd9szDBMClient._getcCs
t��jS)N)�uuid�uuid1�hexr.r.r.r/�_idCsz
DBMClient._idcCs$|��}|j|d<tf|�|_|jS)z1
        Get saved task
        :return:
        r$)rdr$rr�)r,Z
saved_taskr.r.r/�get_taskGs
zDBMClient.get_task)�taskr6cCs2|��|_|j|_d|_|�|���||_|jS)a:
        Create new task and get unique ID
            url --> URL
            client_ip --> IP
            tracing_by --> time|request_qty
            tracing_count --> COUNT
            ini_location --> PATH
            status --> processing
        :param task: a Task instance
        :return: task ID
        �hold)r�r$rEr]�as_dictr�)r,r�r.r.r/�createQs
zDBMClient.create)rwr6cCs6|jdkr|��d|j_||j_|�|j���dS)z�
        Update started|continued task
            status --> running
            starttime --> new timestamp
        :return:
        N�running)r�r�rErwr]r�)r,rwr.r.r/r9ds

zDBMClient.update)�countr6cCs6|jdkr|��d|j_||j_|�|j���dS)z}
        Update stopped task
            status --> stopped
            tracing_count --> new value
        :return:
        N�stopped)r�r�rErvr]r�)r,r�r.r.r/�stopqs

zDBMClient.stopcCs.|jdkr|��d|j_|�|j���dS)zY
        Complete tracing task
            status --> completed
        :return:
        N�	completed)r�r�rEr]r�)r,r.r.r/�complete~s
zDBMClient.completecCs|j|j=dS)z6
        Delete tracing task
        :return:
        N)r�r$)r,r.r.r/r^�szDBMClient.delete)r2rmrnrorpr0r��staticmethodr�rQr]rdr�rr�r��intr9r�r�r^r�r.r.)r1r/r�s



r�cseZdZdZeed�dd��Zed��fdd�Zedd�d	d
�Z	edd�dd�Z
dd�d
d�Zdd�dd�Ze
ed�dd��Zeed�dd�Zedd�dd�Zedd�dd�Zdd�eedd�dd�Zedd�d d!�Zdd�d"d#�Zdd�d$d%�Zdd�d&d'�Z�ZS)(�	APIClientz%
    X-Ray task API client class
    )r6cCst�dt�d�S)z%
        Base endpoint: task
        z://z/api/xray/task)r
r)r,r.r.r/r3�szAPIClient.main_endpointcs$|jdkrt�j|jd�St���S)z�
        Construct query string
        :return: string either including system_id and task_id
                or system_id only
        r)r)r$r�r?)r,r=)r1r.r/r?�s
zAPIClient._query_stringN)r�r6cCs<|j�d|����}|�|dd�|��D��}|d|_dS)z�
        POST request to "create a task" API endpoint with given data
        Obtains a task ID
        :param post_data: dict with POST data
        z/create?cSsi|]\}}|dkr||�qS)rwr.)rr�r�r.r.r/r��sz*APIClient._post_create.<locals>.<dictcomp>rN)r3r?r]r�r$)r,r�rTZ
response_datar.r.r/�_post_create�szAPIClient._post_createcCs$|j�d|����}|�||�dS)z|
        POST request to "update a task" API endpoint with given data
        :param post_data: dict with POST data
        z/update?N)r3r?r])r,r�rTr.r.r/�_post_update�szAPIClient._post_updatecCs.|jdd�}|�d|����}|�|�dS)z<
        GET request to "share a task" API endpoint
        N���z/share-request?)r3r?rd)r,Zshare_endpointrTr.r.r/�_share�szAPIClient._sharecCs"|j�d|����}|�|�dS)z>
        POST request to "delete a task" API endpoint
        z/delete?N)r3r?r])r,rTr.r.r/r_�szAPIClient._deletecCs|�|���S)z1
        Get saved task
        :return:
        )r�rd)r,r.r.r/r��szAPIClient.get_task)r�r6cs,d|_���fdd�|����D���jS)aH
        Create new task and get unique ID
            url --> URL
            client_ip --> IP
            tracing_by --> time|request_qty
            tracing_count --> COUNT
            ini_location --> PATH
            status --> processing
        :param task: a Task instance
        :return: updated Task instance
        r�cs i|]\}}|�jkr||�qSr.)r})rr�r�)r,r.r/r��sz$APIClient.create.<locals>.<dictcomp>)rEr�r�r�r$)r,r�r.)r,r/r��s zAPIClient.create)rwr6cCs|�d|d��dS)z�
        Update started|continued task
            status --> running
            starttime --> new timestamp
        :param starttime: time of starting the Task
        r�)rErwN)r�)r,rwr.r.r/r9�szAPIClient.update)r�r6cCs|�d|i�dS)zw
        Update tracing_count only. No status updated
            tracing_count --> new value
        :return:
        rvN)r�)r,r�r.r.r/�update_count_only�szAPIClient.update_count_only)rv)rzrvr6cCs*|dkrd|i}n
||d�}|�|�dS)a
        Update tracing_count only. No status updated
            request_count --> new value
            tracing_count --> new value if given
        :param request_count: number of requests already traced
        :param tracing_count: number of requests left to trace
        Nrz)rzrv)r�)r,rzrvrRr.r.r/�update_counts_only�s
	
zAPIClient.update_counts_onlycCs|�d|d��dS)z}
        Update stopped task
            status --> stopped
            tracing_count --> new value
        :return:
        r�)rErvN)r�)r,r�r.r.r/r�szAPIClient.stopcCs|�ddi�dS)zY
        Complete tracing task
            status --> completed
        :return:
        rEr�N)r�)r,r.r.r/r�szAPIClient.completecCs|��dS)z5
        Share tracing task
        :return:
        N)r�)r,r.r.r/�shareszAPIClient.sharecCs|��dS)z6
        Delete tracing task
        :return:
        N)r_)r,r.r.r/r^szAPIClient.delete)r2rmrnrorqrpr3r?rQr�r�r�r_rrr�r�r�r9r�r�r�r�r�r^r�r.r.)r1r/r��s$

	
r�c@s2eZdZdZeed�dd��Zedd�dd�ZdS)	�
SendClientz)
    X-Ray requests API client class
    )r6cCst�dt�d�S)z)
        Base endpoint: requests
        z://z/api/xray/requests)r
r)r,r.r.r/r3+szSendClient.main_endpointN)rRr6cCs(|j�d|����}|j||dd�dS)zS
        Send given data to ClickHouse
        :param data: dict with data
        r`F)rVN)r3r?r])r,rRrTr.r.r/�__call__2szSendClient.__call__)	r2rmrnrorqrpr3rQr�r.r.r.r/r�&sr�csheZdZdZeed�dd��Zed��fdd�Zed�dd�Z	eed	�d
d�Z
eeed�d
d�Z�Z
S)�UIAPIClientz,
    X-Ray User plugin API client class
    )r6cCst�dt�d�S)z)
        Base endpoint: requests
        z://z	/api/xray)r
r)r,r.r.r/r3@szUIAPIClient.main_endpointcs dd�|��D�}t�jf|�S)z�
        Construct query string
        :return: string including system_id
                 and given kwargs, filtered by non-empty values
        cSsi|]\}}|dk	r||�qS)Nr.)rr�r�r.r.r/r�Msz-UIAPIClient._query_string.<locals>.<dictcomp>)r�r�r?)r,r=Zfilter_empty)r1r.r/r?GszUIAPIClient._query_stringc
Cs�|jt�d�}|j�d|��}|�|�}xt|dD]h}|ddkrDq2tj|d��dd���}t|��.}|d	d
kr�|d|j	|d<|j	|d
<WdQRXq2W|S)zd
        Get list of tasks and return not processed (full) response
        from API server
        )r|z/tasks?rNrEr�rr")�digest_sizeru�timeryrvrzN)
r?rr3rc�hashlib�blake2b�encode�	hexdigestr�processed_requests)r,�qsrTr@r��fake_id�storager.r.r/�
get_task_listPs

zUIAPIClient.get_task_list)r$r6cCs&|j|d�}|j�d|��}|�|�S)zG
        Get list of requests collected for given tracing task
        )rz
/requests?)r?r3rc)r,r$r�rTr.r.r/�get_request_listfszUIAPIClient.get_request_list)r$�
request_idr6cCs(|j||d�}|j�d|��}|�|�S)zU
        Get collected statistics for given request ID of given tracing task
        )rr�z	/request?)r?r3rc)r,r$r�r�rTr.r.r/�get_request_datanszUIAPIClient.get_request_data)r2rmrnrorqrpr3r?rQr�r�r�r�r�r.r.)r1r/r�;s	r�cs�eZdZdZ�fdd�Zeed�dd��Zeed�dd��Z	ed�d	d
�Z
edd�d
d�Zdeee
d�dd�Zedd�dd�Zed�dd�Zed�dd�Zeed�dd�Zee
d�dd�Z�ZS)�SmartAdviceAPIClientz(
    X-Ray Adviser API client class
    cst�jdd�dS)N�
not_needed)r)r�r0)r,)r1r.r/r0}szSmartAdviceAPIClient.__init__)r6cCsdt�d�S)z)
        Base endpoint: requests
        zhttps://z/api)r)r,r.r.r/r3�sz"SmartAdviceAPIClient.main_endpointcCsdS)z3
        Limit fields available for update
        )rE�sourcerJr.)r,r.r.r/�fields_allowed�sz#SmartAdviceAPIClient.fields_allowedcKs6dd�tD�}t�}|r&|�d|f�tjj|dd�S)za
        Construct query string
        :return: string including types and given kwargs
        cSsg|]}d|f�qS)r�r.)rZ_tr.r.r/r��sz6SmartAdviceAPIClient._query_string.<locals>.<listcomp>�username�,)�safe)�supported_advice_typesr�appendr:r;r<)r,r=r>�user_contextr.r.r/r?�s
z"SmartAdviceAPIClient._query_stringN)rRr6cCs"|j�d�}|j||ddd�dS)z]
        Send given data to Adviser microservice
        :param data: dict with data
        z
/requests/addF)rVrAN)r3r])r,rRrTr.r.r/r��s
zSmartAdviceAPIClient.__call__)rTrUr6c
Csj|j�d|�y|jj|||jd�}Wn4tk
rZ}z|�||dd�|�Wdd}~XYnX|j|dd�S)z�
        Perform PATCH request to given endpoint.
        Add payload as JSON.
        :param endpoint: target URL
        :param payload: dict with data to PATCH
        :return: full response
        zSending PATCH request to %s)rMrWzPATCH failedz/Failed to PATCH data to Smart Advice API serverNF)rA)r'rKr)�patchr8rrZrO)r,rTrUr[r\r.r.r/�_patch�s
zSmartAdviceAPIClient._patchcCs |j�d�}|j||dd�dS)z9
        Send statistics to Adviser microservice
        z/requests/metadataF)rAN)r3r])r,rRrTr.r.r/�	send_stat�szSmartAdviceAPIClient.send_statcCs"|j�d|����}|j|dd�S)z$
        Get list of advice
        z
/advice/list?F)rA)r3r?rc)r,rTr.r.r/�advice_list�sz SmartAdviceAPIClient.advice_listcCs|j�d|��}|j|dd�S)z>
        Get urls/advices information per user`s site
        z/advice/site_info/F)rA)r3rc)r,r�rTr.r.r/�	site_info�szSmartAdviceAPIClient.site_info)�	advice_idr6cCs |j�d|�d�}|j|dd�S)z=
        Get details of an advice by given advice_id
        z/advice/z/detailsF)rA)r3rc)r,r�rTr.r.r/�advice_details�sz#SmartAdviceAPIClient.advice_detailscs2�fdd�|��D�}�j�d|��}��||�S)z�
        Partial update of an advice by given advice_id.
        Fields allowed for update are limited by fields_allowed property
        cs i|]\}}|�jkr||�qSr.)r�)rr�r�)r,r.r/r��sz6SmartAdviceAPIClient.update_advice.<locals>.<dictcomp>z/advice/)r�r3r�)r,r�r=rRrTr.)r,r/�
update_advice�sz"SmartAdviceAPIClient.update_advice)N)r2rmrnror0rqrpr3r�r�r?rQr�rr�r�rr�r�r�r�r�r�r.r.)r1r/r�xs	r�cs�eZdZdZ�fdd�Zed�dd�Zeed�dd��Zde	e
ed
�dd�Zeeed
�dd�Z
eeed
�dd�Zeeed
�dd�Zed�dd�Z�ZS)�AWPProvisionAPIClientz(
    X-Ray Adviser API client class
    cst�jdd�dS)Nr�)r)r�r0)r,)r1r.r/r0�szAWPProvisionAPIClient.__init__)r6cKstj�|�S)N)r:r;r<)r,r=r.r.r/r?�sz#AWPProvisionAPIClient._query_stringcCsdt�d�S)z)
        Base endpoint: requests
        zhttps://z/awp)r)r,r.r.r/r3�sz#AWPProvisionAPIClient.main_endpointT)r@rAr6cCs|j||d�S)N)rA)rO)r,r@rAr.r.r/rS�sz'AWPProvisionAPIClient._process_response)�
account_id�domain�websitecCs&|j�d�}|j||||d�ddd�S)zG
        Gets pullzone if already exists, otherwise creates it
        z
/cdn/pullzone)r��original_urlr�F)rVrA)r3r])r,r�r�r�rTr.r.r/�get_create_pullzone�sz)AWPProvisionAPIClient.get_create_pullzonecCs2|j�d�}|j|�d|j|||d���ddd�S)zG
        Gets pullzone if already exists, otherwise creates it
        z
/cdn/pullzoner`)r�r�r�F)rVrA)r3r_r?)r,r�r�r�rTr.r.r/�remove_pullzone�sz%AWPProvisionAPIClient.remove_pullzonecCs&|j�d�}|j||||d�ddd�S)Nz
/cdn/purge)r�r�r�F)rVrA)r3r])r,r�r�r�rTr.r.r/�purge_cdn_cachesz%AWPProvisionAPIClient.purge_cdn_cache)r�cCs"|j�d|j|d���}|�|�S)Nz/cdn/usage?)r�)r3r?rd)r,r�rTr.r.r/�	get_usageszAWPProvisionAPIClient.get_usage)T)r2rmrnror0rpr?rqr3rrrrQrSr�r�r�r�r�r.r.)r1r/r��s	r�)1ror�r�rMr%�urllib.parser:r��	functoolsr�typingrr�requestsrrZrequests.adaptersrZrequests.exceptionsrZ$requests.packages.urllib3.util.retryr	Zadviser.advice_typesrr�Zinternal.constantsrr
rZinternal.exceptionsrZinternal.local_countersrZinternal.typesrZinternal.user_plugin_utilsrrZinternal.utilsrrrsr�r�r�r�r�r�r�r.r.r.r/�<module>
s<Et=e