/usr/lib/python3/dist-packages/landscape/lib/__pycache__
Edit: /usr/lib/python3/dist-packages/landscape/lib/__pycache__/fetch.cpython-310.pyc (7235B)
o
c @ s d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZ d dl m
Z
m Z G dd de Z
G dd d e
ZG d
d d e
Zd d
i ddddd dddf ddZdd Zd ddZd ddZd ddZdd Zed krteejd d dS dS )! N) OptionParser) DeferredList)
deferToThread) iteritems
networkStringc @ s e Zd ZdS )
FetchErrorN)__name__
__module__ __qualname__ r r 5/usr/lib/python3/dist-packages/landscape/lib/fetch.pyr s r c @ s$ e Zd Zdd Zdd Zdd ZdS )
HTTPCodeErrorc C || _ || _d S N) http_codebody)selfr r r r r __init__
zHTTPCodeError.__init__c C
d| j S )Nz Server returned HTTP code %dr r r r r __str__
zHTTPCodeError.__str__c C r )Nz
r r r r r __repr__ r zHTTPCodeError.__repr__N)r r r
r r r r r r r r
s r
c @ s0 e Zd Zdd Zdd Zdd Zedd Zd S )
PyCurlErrorc C r r )
error_code_message)r r messager r r r r zPyCurlError.__init__c C d| j | jf S )Nz Error %d: %sr r r r r r r # s zPyCurlError.__str__c C r )Nz r r r r r r & s zPyCurlError.__repr__c C s | j S r )r r r r r r * s zPyCurlError.messageN)r r r
r r r propertyr r r r r r s r F iX Tc
C s ddl } t|ts|d}t|}
t }|du r | }|| jt t
| |rF|| j d |rF|| j t
| || j|
j |rV| drV|| jt | |rh|| jdd tt|D |rq|| jd | rz|| jd |
dur|| jt |
| dur|| jt | || jd
|| j| || jd || j | || j d || j |j || j d || j!d z|" W n | j#y } z t$|j%d |j%d d}~ww |& }|'| j(}|d krt)|||S )
aC Retrieve a URL and return the content.
@param url: The url to be fetched.
@param post: If true, the POST method will be used (defaults to GET).
@param data: Data to be sent to the server as the POST content.
@param headers: Dictionary of header => value entries to be used on the
request.
@param curl: A pycurl.Curl instance to use. If not provided, one will be
created.
@param cainfo: Path to the file with CA certificates.
@param insecure: If true, perform curl using insecure option which will
not attempt to verify authenticity of the peer's certificate. (Used
during autodiscovery)
@param follow: If True, follow HTTP redirects (default True).
@param user_agent: The user-agent to set in the request.
@param proxy: The proxy url to use for the request.
r Nzutf-8Tzhttps:c S s g | ]}d | qS )z%s: %sr ).0pairr r r
\ s zfetch..F s gzip,deflate )*pycurl
isinstancebytesencodeioBytesIOCurlsetoptURLr strPOST
POSTFIELDSIZElen READFUNCTIONread
startswithCAINFO
HTTPHEADERsortedr SSL_VERIFYPEERFOLLOWLOCATION USERAGENTPROXY MAXREDIRSCONNECTTIMEOUTLOW_SPEED_LIMITLOW_SPEED_TIMENOSIGNAL
WRITEFUNCTIONwriteDNS_CACHE_TIMEOUTENCODINGperformerrorr argsgetvaluegetinfo HTTP_CODEr
)urlpostdataheaderscainfocurlconnect_timeout
total_timeoutinsecurefollow
user_agentproxyr* outputinputer r r r r fetch/ s\
r_ c O s t tg| R i |S )z]Retrieve a URL asynchronously.
@return: A C{Deferred} resulting in the URL content.
)r r_ )rL kwargsr r r fetch_async s ra c K sV g }| D ] }t |fi |}|r||| |r ||| || qt|dddS )a9
Retrieve a list of URLs asynchronously.
@param callback: Optionally, a function that will be fired one time for
each successful URL, and will be passed its content and the URL itself.
@param errback: Optionally, a function that will be fired one time for each
failing URL, and will be passed the failure and the URL itself.
@return: A C{DeferredList} whose callback chain will be fired as soon as
all downloads have terminated. If an error occurs, the errback chain
of the C{DeferredList} will be fired immediatly.
T)fireOnOneErrback
consumeErrors)ra addCallback
addErrbackappendr )urlscallbackerrbackr` resultsrP resultr r r fetch_many_async s rl c C s. | ddd }|durtj||}|S )zReturn the last component of the given C{url}.
@param url: The URL to get the filename from.
@param directory: Optionally a path to prepend to the returned filename.
@note: Any trailing slash in the C{url} will be removed
/N)rstripsplitospathjoin)rP directoryfilenamer r r url_to_filename s rv c s. fdd}fdd}t | f||d|S )aJ
Retrieve a list of URLs and save their content as files in a directory.
@param urls: The list URLs to fetch.
@param directory: The directory to save the files to, the name of the file
will equal the last fragment of the URL.
@param logger: Optional function to be used to log errors for failed URLs.
c s, t | d}t|d}|| | d S )Nrt wb)rv openrG close)rR rP ru fdrw r r rG s
z fetch_to_files..writec s r
d|t | jf | S )Nz Couldn't fetch file from %s (%s))r3 value)failurerP )loggerr r log_error s
z!fetch_to_files..log_error)rh ri )rl )rg rt r~ r` rG r r )rt r~ r fetch_to_files s
r c C sZ t }|jddd |jddd |d || \}\}tt||j|j|jd d S ) Nz--post
store_true)actionz--datar" )defaultz--cainfo)rQ rR rT )r
add_option
parse_argsprintr_ rQ rR rT )rL parseroptionsrP r r r test s
r __main__r' )NNr )rq sysr. optparser twisted.internet.deferr twisted.internet.threadsr twisted.python.compatr r Exceptionr r
r r_ ra rl rv r r r argvr r r r s,
R