Yc @ sS d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z m Z m
Z
m Z m Z m
Z
m Z m Z m Z m Z m Z m Z m Z d d l m Z d d l m Z d d l m Z d d l m Z m Z m Z m Z m Z m! Z! m" Z" m# Z# m$ Z$ m% Z% m& Z& m' Z' m( Z( m) Z) d d l m* Z* d d l+ m, Z, d d
l- m. Z. d d l- m/ Z/ d d l0 m1 Z1 e j2 d
Z3 e j2 d e j4 Z5 e j2 d Z6 e j2 d e j4 j7 Z8 d j9 Z: d d d d g Z; d Z< d Z= d Z> e? d Z@ e? d ZA e? d ZB e? e e? d ZC e? d ZD d ZE e j2 d e j4 ZF eE d ZG d! e jH d" e
d# d$ jH f ZI d% eJ f d& YZK d' eK f d( YZL d e f d) YZM e j2 d* jN ZO d+ ZP d, ZQ d- ZR d d. ZS d/ ZT e jU d0 ZV eS e< eV ZV d1 ZW d2 ZX d S(3 s# PyPI and direct package downloadingiN(
t
CHECKOUT_DISTt Distributiont BINARY_DISTt normalize_patht SOURCE_DISTt requiret Environmentt find_distributionst safe_namet safe_versiont to_filenamet Requirementt DEVELOP_DIST( t ssl_support( t log( t DistutilsError( t urllib2t httplibt StringIOt HTTPErrort urlparset
urlunparset unquotet splitusert url2pathnamet name2codepointt unichrt urljoint urlsplitt
urlunsplit( t filterfalse( t translate( t hashlib( t wraps( t get_all_headerss ^egg=([-A-Za-z0-9_.]+)$s href\s*=\s*['"]?([^'"> ]+)s ([^<]+)
\s+\(md5\)s ([-+.a-z0-9]{2,}):s .tar.gz .tar.bz2 .tar .zip .tgzt PackageIndext distros_for_urlt parse_bdist_wininstt interpret_distro_namei c C s | j } d \ } } } | j d r | j d rL | d } d } q | j d d r~ | d d !} | d } d } q | j d r | d
} d } q | j d d
r | d d !} | d
} d } q n | | | f S( s= Return (base,pyversion) or (None,None) for possible .exe names .exes
.win32.exeit win32s .win32-pyiiis .win-amd64.exeis win-amd64s
.win-amd64-pyiN( NNN( t lowert Nonet endswitht
startswith( t nameR( t baset py_vert plat( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR% , s$
c C s t | \ } } } } } } t | j d d } | d k rk | d k rk t | j d d } n d | k r | j d d \ } } n | | f S( Nt /is sourceforge.nett downloadit #i ( R R t split( t urlt schemet servert patht
parameterst queryt fragmentR- ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt egg_info_for_urlD s c c s t | \ } } x t | | | D] } | Vq% W| r t j | } | r x1 t | | j d | d t D] } | Vqq Wq n d S( sE Yield egg or source distribution objects that might be found at a URLi t
precedenceN( R; t distros_for_locationt EGG_FRAGMENTt matchR& t groupR ( R4 t metadataR- R: t distR? ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR$ L s "c C s | j d r | d } n | j d rM d | k rM t j | | | g S| j d r t | \ } } } | d k r t | | | | t | Sn x> t D]6 } | j | r | t | } t | | | Sq Wg S( s: Yield egg or source distribution objects based on basenames .egg.zipis .eggt -s .exeN( R* R t
from_locationR% R) R& R t
EXTENSIONSt len( t locationt basenameRA t win_baseR. t platformt ext( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR= X s
c C s" t t | t j j | | S( sE Yield possible egg or source distribution objects based on a filename( R= R t osR7 RH ( t filenameRA ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt distros_for_filenamem s c c s | j d } | s^ xF t | d D]1 \ } } t | d k r& | j d r& d Sq& Wn xa t d t | d D]F } t | | d j | | d j | | d | d | d | Vqx Wd S(
s Generate alternative interpretations of a source distro name
Note: if `location` is a filesystem filename, you should call
``pkg_resources.normalize_path()`` on it before passing it to this
routine!
RC i i s py2.Ni t
py_versionR< RJ ( R3 t enumerateRF R+ t rangeR t join( RG RH RA RO R< RJ t partst it p( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR& t s ! ) c c s t } | j } | d k rM xg t | j | D] } | | | Vq1 Wn; x8 | D]0 } | | } | | k rT | | | VqT qT Wd S( sH List unique elements, preserving order. Remember all elements ever seen.N( t sett addR) R t __contains__( t iterablet keyt seent seen_addt elementt k( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt unique_everseen s
c s t f d } | S( ss
Wrap a function returning an iterable such that the resulting iterable
only ever yields unique items.
c s t | | S( N( R_ ( t argst kwargs( t func( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt wrapper s ( R! ( Rb Rc ( ( Rb s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt
unique_values s s( <([^>]*\srel\s*=\s*['"]?([^'">]+)[^>]*)>c c s x t j | D] } | j \ } } t t t j | j j d } d | k sg d | k r x7 t j | D]# } t
| t | j d Vqw Wq q Wxd d D]\ } | j
| } | d k r t j | | } | rt
| t | j d Vqq q Wd S(
sE Find rel="homepage" and rel="download" links in `page`, yielding URLst ,t homepageR1 i s
| Home Pages | Download URLiN( s
| Home Pages | Download URL( t RELt finditert groupsRV t mapt strt stripR( R3 t HREFR t
htmldecodeR@ t findt search( R4 t pageR? t tagt relt relst pos( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt find_external_links s '(
s Python-urllib/%s setuptools/%si t
setuptoolsi t ContentCheckerc B s) e Z d Z d Z d Z d Z RS( sP
A null content checker that defines the interface for checking content
c C s d S( s3
Feed a block of data to the hash.
N( ( t selft block( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt feed s c C s t S( sC
Check the hash. Return False if validation fails.
( t True( Ry ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt is_valid s c C s d S( su
Call reporter with information about the checker (hash name)
substituted into the template.
N( ( Ry t reportert template( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt report s ( t __name__t
__module__t __doc__R{ R} R ( ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyRx s t HashCheckerc B sJ e Z e j d Z d Z e d Z d Z d Z d Z
RS( sK (?Psha1|sha224|sha384|sha256|sha512|md5)=(?P[a-f0-9]+)c C sO | | _ | d k r0 t j | d t | _ n t j | | _ | | _ d S( Nt md5t usedforsecurity( t hash_nameR t newt Falset hasht expected( Ry R R ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt __init__ s
c C sL t | d } | s t S| j j | } | s< t S| | j S( s5 Construct a (possibly null) ContentChecker from a URLi( R Rx t patternRp t groupdict( t clsR4 R: R? ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt from_url s c C s | j j | d S( N( R t update( Ry Rz ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR{ s c C s | j j | j k S( N( R t hexdigestR ( Ry ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR} s c C s | | j } | | S( N( R ( Ry R~ R t msg( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s
( R R t ret compileR R t classmethodR R{ R} R ( ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s c B sv e Z d Z d d% d& e d Z e d Z e d Z e d Z d Z
d Z d Z d
Z
d& d Z d Z d& d
Z d Z d Z d Z d Z d Z e e e d& d Z e e d Z d Z d Z d Z d Z d& d Z d Z d Z d Z d Z d Z! e" e d Z# d Z$ d! Z% d" Z& d# Z' d$ Z( RS(' s; A distribution index that scans web pages for download URLss https://pypi.python.org/simplet *c O s t j | | | | d | j d | _ i | _ i | _ i | _ t j d j t
t | j | _
g | _ | r t j r | s t j r t j | | _ n t j | _ d S( NR0 t |( R R R* t index_urlt scanned_urlst fetched_urlst
package_pagesR R RR Rj R R? t allowst to_scanR
t is_availablet find_ca_bundlet
opener_fort openerR t urlopen( Ry R t hostst ca_bundlet
verify_sslR` t kw( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR
s ' !c
C sU | | j k r | r d St | j | %si i i ( R@ ( t m( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt s R N(
Rm Rh R Rn R@ t
ValueErrorRv R; R* t need_version_infot scan_urlt PYPI_MD5t sub(
Ry R4 Rq R R? R R t new_urlR- t frag( ( Ry s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s s$ &
c C s | j d | d S( NsP Page at %s links to .py file(s) without version info; an index scan is required.( t scan_all( Ry R4 ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s c G sO | j | j k r; | r+ | j | | n | j d n | j | j d S( Ns6 Scanning index of all packages (this may take a while)( R R R R R ( Ry R R` ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s
c C s | j | j | j d | j j | j sN | j | j | j d n | j j | j ss | j | n x3 t | j j | j d D] } | j | q Wd S( NR0 ( ( R R t unsafe_nameR R RZ t project_namet not_found_in_indexR ( Ry t requirementR4 ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt
find_packages s %c C sk | j | j | x8 | | j D]) } | | k r; | S| j d | | q% Wt t | j | | S( Ns %s does not match %s( t prescanR RZ R t superR# t obtain( Ry R t installerRB ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s
c C si | j | j d | | j se | j t j | t d | j j t j j
| f n d S( s-
checker is a ContentChecker
s Validating %%s checksum for %ss7 %s validation failed for %s; possible download problem?N( R R R} R RL t unlinkR R R, R7 RH ( Ry t checkerRM t tfp( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt
check_hash s
c C sr xk | D]c } | j d k sJ t | sJ | j d sJ t t | rZ | j | q | j j | q Wd S( s; Add `urls` to the list that will be prescanned for searchess file:N( R R) R R+ R R$ R t append( Ry t urlsR4 ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt add_find_links s
c C s2 | j r% t t | j | j n d | _ d S( s7 Scan urls scheduled for prescanning (e.g. --find-links)N( R R Rj R R) ( Ry ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s c C sN | | j r | j d } } n | j d } } | | | j | j d S( Ns# Couldn't retrieve index page for %rs3 Couldn't find index page for %r (maybe misspelled?)( RZ R R R R ( Ry R t methR ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s
c C s t | t s t | } | r| | j | j d | | } t | \ } } | j d rx | j | | | } n | St j j
| r | Sy t j | } Wq t k
r t
d | f q Xn t | j | | d d S( s Locate and/or download `spec` to `tmpdir`, returning a local path
`spec` may be a ``Requirement`` object, or a string containing a URL,
an existing local filename, or a project/version requirement spec
(i.e. the string form of a ``Requirement`` object). If it is the URL
of a .py file with an unambiguous ``#egg=name-version`` tag (i.e., one
that escapes ``-`` as ``_`` throughout), a trivial ``setup.py`` is
automatically created alongside the downloaded file.
If `spec` is a ``Requirement`` object or a string containing a
project/version requirement spec, this method returns the location of
a matching distribution (possibly after downloading it to `tmpdir`).
If `spec` is a locally existing file or directory name, it is simply
returned unchanged. If `spec` is a URL, it is downloaded to a subpath
of `tmpdir`, and the local filename is returned. Various errors may be
raised if a problem occurs during downloading.
i s .pys1 Not a URL, existing file, or requirement spec: %rRG N( R R R t
_download_urlR@ R; R* t gen_setupRL R7 R t parseR R R t fetch_distributionR) ( Ry t spect tmpdirR5 t foundR- R: ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR1 s"
c sK j d | i d } d f d } | rc j j | | | } n | d k r | p | | | } n | d k r j d k r j | | } n | d k r | r j | | | } n | d k r j d r d pd | n/ j d | | j d j | j | Sd S( s| Obtain a distribution suitable for fulfilling `requirement`
`requirement` must be a ``pkg_resources.Requirement`` instance.
If necessary, or if the `force_scan` flag is set, the requirement is
searched for in the (online) package index as well as the locally
installed packages. If a distribution matching `requirement` is found,
the returned distribution's ``location`` is the value you would have
gotten from calling the ``download()`` method with the matching
distribution's URL or filename. If no matching distribution is found,
``None`` is returned.
If the `source` flag is set, only source distributions and source
checkout links will be considered. Unless the `develop_ok` flag is
set, development and system eggs (i.e., those using the ``.egg-info``
format) will be ignored.
s Searching for %sc s | d k r } n x | | j D]q } | j t k rn rn | k r# j d | d | R? R& R@ R) t versionRF RL R7 RH t dirnameRR t setuptools.command.easy_installR t shutilt copy2R t writeR t splitextR R ( Ry RM R: R R? t dR RH t dstR R ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR h s4 4
i c
C s | j d | d \ } } } zIt j | } | j | } t | t rq t d | | j | j f n | j } d } | j
} d }
d | k r t | d } t t
t | }
| j | | | | |
n t | d } x` t rO| j | } | rK| j | | j | | d 7} | j | | | | |
q Pq W| j | | | | SWd | r~| j n | r| j n Xd S(
Ns Downloading %ss Can't download %s: %s %si is content-lengths Content-Lengtht wbi ( NNN( R R) R R R R R R R R t dl_blocksizeR" t maxRj t intt
reporthookR R| R R{ R
R R (
Ry R4 RM t fpR R R R t blocknumt bst sizet sizesRz ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt _download_to s>
c C s d S( N( ( Ry R4 RM R t blksizeR ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s c C s | j d r t | Sy t | | j SWnt t j f k
r t j d } d j g | j
D] } t | ^ qf } | r | j | | qt
d | | f n't j k
r t j d } | St j k
r/t j d } | r| j | | j qt
d | | j f n t j k
rt j d } | rk| j | | j qt
d | | j f nS t j k
rt j d } | r| j | | qt
d | | f n Xd S( Ns file:i t s %s %ss Download error for %s: %ss; %s returned a bad status line. The server might be down, %s( R+ t
local_opent open_with_authR R R t
InvalidURLt syst exc_infoRR R` Rk R R R R t URLErrort reasont
BadStatusLinet linet
HTTPException( Ry R4 t warningt vt argR ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s@
+c C s< t | \ } } | rL x7 d | k rH | j d d j d d } q Wn d } | j d rn | d } n t j j | | } | d k s | j d r | j | | S| d
k s | j d r | j | | S| j d r | j | | S| d
k rt
t | d S| j | t
| j | | Sd S( Ns ..t .s \t _t __downloaded__s .egg.zipit svns svn+t gits git+s hg+R i ( R; t replaceR* RL R7 RR R+ t
_download_svnt
_download_gitt _download_hgR R R R| t _attempt_download( Ry R5 R4 R R, R: RM ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s$ %
c C s | j | t d S( N( R R| ( Ry R4 ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s c C sK | j | | } d | j d d j k rC | j | | | S| Sd S( NR s content-typeR ( R R R( t _download_html( Ry R4 RM R ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR1 s c C s t | } xW | D]O } | j r t j d | r^ | j t j | | j | | SPq q W| j t j | t d | d S( Ns ([^- ]+ - )?Revision \d+:s Unexpected HTML page found at ( R Rl R Rp R RL R R. R ( Ry R4 R RM R R# ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR2 s
c
C sT | j d d d } d } | j j d r#d | k r#t | \ } } } } } } | r#| j d r#d | d k r#| d j d d \ } } t | \ }
} |
r d
|
k r |
j d
d \ } }
d | |
f } n
d |
} | } t | | | | | | f } q q#n | j d
| | t j d | | | f | S( NR2 i i R s svn:t @s //R0 i t :s --username=%s --password=%ss --username=s' Doing subversion checkout from %s to %ss svn checkout%s -q %s %s( R3 R( R+ R R R R RL t system( Ry R4 RM t credsR5 t netlocR7 RU t qR t autht hostt usert pw( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR. s" ! &
'c C s t | \ } } } } } | j d d d } | j d d d } d } d | k rt | j d d \ } } n t | | | | d f } | | f S( Nt +i iR2 i R3 R ( R R3 R) t rsplitR ( R4 t
pop_prefixR5 R7 R7 R9 R t rev( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt _vcs_split_rev_from_url, s c C s | j d d d } | j | d t \ } } | j d | | t j d | | f | d k r | j d | t j d | | f n | S( NR2 i i R? s Doing git clone from %s to %ss git clone --quiet %s %ss Checking out %ss" (cd %s && git checkout --quiet %s)( R3 RA R| R RL R5 R) ( Ry R4 RM R@ ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR/ > s c C s | j d d d } | j | d t \ } } | j d | | t j d | | f | d k r | j d | t j d | | f n | S( NR2 i i R? s Doing hg clone from %s to %ss hg clone --quiet %s %ss Updating to %ss (cd %s && hg up -C -r %s >&-)( R3 RA R| R RL R5 R) ( Ry R4 RM R@ ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR0 N s c G s t j | | d S( N( R R ( Ry R R` ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR ^ s c G s t j | | d S( N( R R ( Ry R R` ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR a s c G s t j | | d S( N( R R ( Ry R R` ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR d s ( R N() R R R R) R| R R R R R R R R R R R R R R R R R1 R R R R R R R R R R1 R2 R. t staticmethodRA R/ R0 R R R ( ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR# sL
0
* )A
* $ ( s! &(#(\d+|x[\da-fA-F]+)|[\w.:-]+);?c C s3 t | t s | S| d k r) t | St | S( Ni ( R R R t chr( t c( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt uchrk s
c C s{ | j d } | j d r4 t | d d } n= | j d rV t | d } n t j | | j d } t | S( Ni s #xi i R2 i ( R@ R+ R R R RE ( R? t what( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt
decode_entityq s c C s
t t | S( s' Decode HTML entities in the given text.( t
entity_subRG ( t text( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyRn { s c s f d } | S( Nc s f d } | S( Nc s? t j } t j z | | SWd t j | Xd S( N( t sockett getdefaulttimeoutt setdefaulttimeout( R` Ra t old_timeout( Rb t timeout( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt _socket_timeout s
( ( Rb RO ( RN ( Rb s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyRO s ( ( RN RO ( ( RN s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt socket_timeout s c C s= t | } | j } t j | } | j } | j S( s
A function compatible with Python 2.3-3.3 that will encode
auth from a URL suitable for an HTTP header.
>>> _encode_auth('username%3Apassword')
u'dXNlcm5hbWU6cGFzc3dvcmQ='
( R t encodet base64t encodestringR t rstrip( R9 t auth_st
auth_bytest
encoded_bytest encoded( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt _encode_auth s
c C sQ t | \ } } } } } } | j d r? t j d n | d k r` t | \ } } n d } | r d t | } t | | | | | | f }
t j |
} | j
d | n t j | } | j
d t | | } | rMt | j \ }
} } } } } |
| k rM| | k rMt |
| | | | | f | _ qMn | S(
s4 Open a urllib2 request, handling HTTP authenticationR4 s nonnumeric port: ''t httpt httpss Basic t
Authorizations
User-Agent( RZ R[ N(
R R* R R R R) RY R R t Requestt
add_headert
user_agentR4 ( R4 R R5 R7 R7 t paramsR9 R R9 R: R t requestR t s2t h2t path2t param2t query2t frag2( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyR s& !'c C s | S( N( ( R4 ( ( s< /usr/lib/python2.7/site-packages/setuptools/package_index.pyt
fix_sf_url s c C so t | \ } } } } } } t | } t j j | rI t j | S| j d r=t j j | r=g } x t j | D] } | d k r t
t j j | | d }
|
j } |
j
Pn. t j j t j j | | r | d 7} n | j d | | f q Wd | d d j | } d \ } }
n d \ } }
} t | | |
i d
d 6t | S( s7 Read a local path, with special support for directoriesR0 s
index.htmlt rbs <a href=%r>%s</a>s <html><head><title>%ss %s |