IPFire-SCM

ipfire-scm@lists.ipfire.org

1 participants
7826 discussions

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. 0bbad22ef840764d9cb3fbc453f404e3920e9c89
by git＠ipfire.org 19 Mar '10

19 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via 0bbad22ef840764d9cb3fbc453f404e3920e9c89 (commit) via cee45e8438e3734933abc58d31bb63c5dd0db00f (commit) via cdbc3c2819e140c3c3a141daaa1f32748c980773 (commit) via 6b7d4f44d6a4a46f5a41858f02884212a7d31164 (commit) via 092e60f5a7b5fc866f74fb51f9e7e221a0c88711 (commit) via 51d2e4a977c59d01f62b0415c4ec195dc97aee13 (commit) via 3c2c1737645490df48528ba4a663e968c65478ce (commit) via d1ece662305f6e778ce0ba2f0ebc03e058bcca1b (commit) via 698742c328b40c9037e85b4a258fd41a51e32108 (commit) via f62fe495e237da299fa3beda87edbe9d196398a9 (commit) via 46c7e34e14394dff715c6cc49cecc45a5590a4a5 (commit) from 58a60fde7432a8276ec821019d401fa6e0e46dc4 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 0bbad22ef840764d9cb3fbc453f404e3920e9c89 Merge: cee45e8438e3734933abc58d31bb63c5dd0db00f cdbc3c2819e140c3c3a141daaa1f32748c980773 Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 15:40:53 2010 +0100 Merge commit 'stevee/toolchain' commit cee45e8438e3734933abc58d31bb63c5dd0db00f Merge: 51d2e4a977c59d01f62b0415c4ec195dc97aee13 58a60fde7432a8276ec821019d401fa6e0e46dc4 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 15:26:42 2010 +0100 Merge branch 'master' of ssh://git.ipfire.org/pub/git/ipfire-3.x into glibc-locale-fix Conflicts: pkgs/core/ccache/ccache.nm commit cdbc3c2819e140c3c3a141daaa1f32748c980773 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 15:19:05 2010 +0100 expect: Remove broken patches from toolchain. commit 6b7d4f44d6a4a46f5a41858f02884212a7d31164 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 15:15:20 2010 +0100 libpcap: Update to latest git revision. commit 092e60f5a7b5fc866f74fb51f9e7e221a0c88711 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 15:14:23 2010 +0100 btrfs-progs: Clean naoki-makefile. commit 51d2e4a977c59d01f62b0415c4ec195dc97aee13 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 14:58:02 2010 +0100 glibc: Fix broken locale files. As a fact that we are only supporting a few languages atm I´ve disabled the other ones. This fixes issure #589. commit 3c2c1737645490df48528ba4a663e968c65478ce Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 14:48:23 2010 +0100 m4: Fix testsuite. This fixes issure #588. commit d1ece662305f6e778ce0ba2f0ebc03e058bcca1b Merge: 698742c328b40c9037e85b4a258fd41a51e32108 865a945a4b5895c08b3a7aba885b7f7462e87c01 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 14:12:38 2010 +0100 Merge branch 'master' of ssh://git.ipfire.org/pub/git/ipfire-3.x into clean commit 698742c328b40c9037e85b4a258fd41a51e32108 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 09:59:28 2010 +0100 ccache: Clean up naoki-makefile. commit f62fe495e237da299fa3beda87edbe9d196398a9 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 09:55:03 2010 +0100 cairo: Build devel package. commit 46c7e34e14394dff715c6cc49cecc45a5590a4a5 Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Fri Mar 19 09:44:31 2010 +0100 bzip2: Clean up naoki-makefile. ----------------------------------------------------------------------- Summary of changes: pkgs/core/btrfs-progs/btrfs-progs.nm | 14 +----- pkgs/core/bzip2/bzip2.nm | 5 +-- pkgs/core/cairo/cairo.nm | 2 + pkgs/core/ccache/ccache.nm | 7 +-- pkgs/core/glibc/SUPPORTED.locales | 21 ++++++++++ pkgs/core/glibc/glibc.nm | 4 ++ pkgs/core/libpcap/libpcap.nm | 2 +- pkgs/core/m4/m4.nm | 7 +-- .../expect/patches/expect-5.43.0-tcl8.5.6.patch | 42 -------------------- .../expect/patches/expect-5.43.0-tcl8.5.patch | 22 ---------- .../patches/expect-5.43.0-tcl_8.5.8_fix-1.patch | 0 11 files changed, 38 insertions(+), 88 deletions(-) create mode 100644 pkgs/core/glibc/SUPPORTED.locales delete mode 100644 pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.6.patch delete mode 100644 pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.patch copy pkgs/{core => toolchain}/expect/patches/expect-5.43.0-tcl_8.5.8_fix-1.patch (100%) Difference in files: diff --git a/pkgs/core/btrfs-progs/btrfs-progs.nm b/pkgs/core/btrfs-progs/btrfs-progs.nm index cb3812d..0955f8d 100644 --- a/pkgs/core/btrfs-progs/btrfs-progs.nm +++ b/pkgs/core/btrfs-progs/btrfs-progs.nm @@ -43,14 +43,6 @@ endef PKG_TARBALL = $(THISAPP).tar.bz2 -############################################################################### -# Installation Details -############################################################################### - -define STAGE_BUILD - cd $(DIR_APP) && make CFLAGS="$(CFLAGS)" $(PARALLELISMFLAGS) -endef - -define STAGE_INSTALL - cd $(DIR_APP) && make bindir=$(BUILDROOT)/sbin mandir=$(BUILDROOT)/usr/share install -endef +STAGE_INSTALL_TARGET += \ + bindir=$(BUILDROOT)/sbin \ + mandir=$(BUILDROOT)/usr/share diff --git a/pkgs/core/bzip2/bzip2.nm b/pkgs/core/bzip2/bzip2.nm index d206936..14e1f59 100644 --- a/pkgs/core/bzip2/bzip2.nm +++ b/pkgs/core/bzip2/bzip2.nm @@ -45,16 +45,13 @@ PKG_TARBALL = $(THISAPP).tar.gz CFLAGS += -fPIC -D_FILE_OFFSET_BITS=64 -############################################################################### -# Installation Details -############################################################################### - define STAGE_PREPARE_CMDS cd $(DIR_APP) && sed -i 's@$ln -s -f $\$$(PREFIX)/bin/@\1@' Makefile cd $(DIR_APP) && sed -e "s/\$$(CC) -shared/& $(CFLAGS)/" -i Makefile-libbz2_so endef define STAGE_BUILD + $(DO_FIX_LIBTOOL) cd $(DIR_APP) && make -f Makefile-libbz2_so $(PARALLELISMFLAGS) cd $(DIR_APP) && make clean diff --git a/pkgs/core/cairo/cairo.nm b/pkgs/core/cairo/cairo.nm index 473891e..c19359d 100644 --- a/pkgs/core/cairo/cairo.nm +++ b/pkgs/core/cairo/cairo.nm @@ -34,6 +34,8 @@ PKG_URL = http://cairographics.org PKG_LICENSE = LGPLv2 MPLv1.1 PKG_SUMMARY = A 2D graphics library. +PKG_PACKAGES += $(PKG_NAME_REAL)-devel + PKG_BUILD_DEPS+= pkg-config PKG_DEPS += fontconfig freetype libpng pixman zlib diff --git a/pkgs/core/ccache/ccache.nm b/pkgs/core/ccache/ccache.nm index 63a30b8..3e9c455 100644 --- a/pkgs/core/ccache/ccache.nm +++ b/pkgs/core/ccache/ccache.nm @@ -45,15 +45,14 @@ endef PKG_TARBALL = $(THISAPP).tar.gz -CONFIGURE_OPTIONS += --mandir=/usr/share +CONFIGURE_OPTIONS += \ + --mandir=/usr/share define STAGE_PREPARE_CMDS cd $(DIR_APP) && sed -e "s/hash_int(&hash, st.st_mtime);//g" -i ccache.c endef -define STAGE_INSTALL - cd $(DIR_APP) && make install DESTDIR=$(BUILDROOT) - +define STAGE_INSTALL_CMDS mkdir -pv $(BUILDROOT)/usr/ccache/bin for i in gcc g++ cc c++; do \ ln -svf ../../bin/ccache $(BUILDROOT)/usr/ccache/bin/$${i}; \ diff --git a/pkgs/core/glibc/SUPPORTED.locales b/pkgs/core/glibc/SUPPORTED.locales new file mode 100644 index 0000000..9003873 --- /dev/null +++ b/pkgs/core/glibc/SUPPORTED.locales @@ -0,0 +1,21 @@ +# This file names the currently supported and somewhat tested locales. +# If you have any additions please file a glibc bug report. +SUPPORTED-LOCALES=\ +da_DK.UTF-8/UTF-8 \ +da_DK/ISO-8859-1 \ +de_DE.UTF-8/UTF-8 \ +de_DE/ISO-8859-1 \ +de_DE@euro/ISO-8859-15 \ +en_GB.UTF-8/UTF-8 \ +en_GB/ISO-8859-1 \ +en_US.UTF-8/UTF-8 \ +en_US/ISO-8859-1 \ +es_ES.UTF-8/UTF-8 \ +es_ES/ISO-8859-1 \ +es_ES@euro/ISO-8859-15 \ +fr_FR.UTF-8/UTF-8 \ +fr_FR/ISO-8859-1 \ +fr_FR@euro/ISO-8859-15 \ +it_IT.UTF-8/UTF-8 \ +it_IT/ISO-8859-1 \ +it_IT@euro/ISO-8859-15 \ diff --git a/pkgs/core/glibc/glibc.nm b/pkgs/core/glibc/glibc.nm index 9741527..d1a4f54 100644 --- a/pkgs/core/glibc/glibc.nm +++ b/pkgs/core/glibc/glibc.nm @@ -163,6 +163,10 @@ define STAGE_PREPARE # Fix a very weird issue with rpath cd $(DIR_APP) && sed -e "s/LDFLAGS-rpath-ORIGIN/LDFLAGS/g" \ -i iconvdata/Makefile + + # Copy our customized SUPPORTED file to build localization files + # only for supported languages + cp -vf $(DIR_SOURCE)/SUPPORTED.locales $(DIR_APP)/localedata/SUPPORTED endef define STAGE_BUILD diff --git a/pkgs/core/libpcap/libpcap.nm b/pkgs/core/libpcap/libpcap.nm index 89549c9..defe9bb 100644 --- a/pkgs/core/libpcap/libpcap.nm +++ b/pkgs/core/libpcap/libpcap.nm @@ -25,7 +25,7 @@ include $(PKGROOT)/Include PKG_NAME = libpcap -PKG_VER = 1.0.0 +PKG_VER = 1.0.0-5.20091201git117cb5 PKG_REL = 0 PKG_MAINTAINER = diff --git a/pkgs/core/m4/m4.nm b/pkgs/core/m4/m4.nm index 26000b5..ee51012 100644 --- a/pkgs/core/m4/m4.nm +++ b/pkgs/core/m4/m4.nm @@ -49,7 +49,6 @@ CONFIGURE_ENVIRONMENT += \ CPPFLAGS="-D_GNU_SOURCE" \ gl_cv_func_printf_directive_n=no \ -# fails -#define STAGE_TEST -# cd $(DIR_APP) && make check -#endef +define STAGE_TEST + cd $(DIR_APP) && make check +endef diff --git a/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.6.patch b/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.6.patch deleted file mode 100644 index fd75c0e..0000000 --- a/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.6.patch +++ /dev/null @@ -1,42 +0,0 @@ -diff -up expect-5.43/exp_command.c_old expect-5.43/exp_command.c ---- expect-5.43/exp_command.c_old 2009-02-10 16:23:48.000000000 +0100 -+++ expect-5.43/exp_command.c 2009-02-10 16:31:38.000000000 +0100 -@@ -2311,12 +2311,11 @@ Tcl_Obj *CONST objv[]; /* Argument objec - /* Historical note: we used "close" long before there was a */ - /* Tcl builtin by the same name. */ - -- Tcl_CmdInfo info; -+ Tcl_CmdInfo* close_info; - Tcl_ResetResult(interp); -- if (0 == Tcl_GetCommandInfo(interp,"close",&info)) { -- info.clientData = 0; -- } -- return(Tcl_CloseObjCmd(info.clientData,interp,objc_orig,objv_orig)); -+ -+ close_info = (Tcl_CmdInfo*) Tcl_GetAssocData (interp, EXP_CMDINFO_CLOSE, NULL); -+ return(close_info->objProc(close_info->objClientData,interp,objc,objv)); - } - - if (chanName) { -@@ -2961,7 +2960,10 @@ Tcl_Obj *CONST objv[]; - /* if successful (i.e., TCL_RETURN is returned) */ - /* modify the result, so that we will handle it specially */ - -- int result = Tcl_ReturnObjCmd(clientData,interp,objc,objv); -+ Tcl_CmdInfo* return_info = (Tcl_CmdInfo*) -+ Tcl_GetAssocData (interp, EXP_CMDINFO_RETURN, NULL); -+ -+ int result = return_info->objProc(return_info->objClientData,interp,objc,objv); - if (result == TCL_RETURN) - result = EXP_TCL_RETURN; - return result; -diff -up expect-5.43/exp_command.h_old expect-5.43/exp_command.h ---- expect-5.43/exp_command.h_old 2009-02-10 17:33:19.000000000 +0100 -+++ expect-5.43/exp_command.h 2009-02-10 17:33:46.000000000 +0100 -@@ -326,3 +326,6 @@ EXTERN ExpState * expDevttyGet _ANSI_ARG - /* generic functions that really should be provided by Tcl */ - EXTERN int expSizeGet _ANSI_ARGS_((ExpState *)); - EXTERN int expSizeZero _ANSI_ARGS_((ExpState *)); -+ -+#define EXP_CMDINFO_CLOSE "expect/cmdinfo/close" -+#define EXP_CMDINFO_RETURN "expect/cmdinfo/return" diff --git a/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.patch b/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.patch deleted file mode 100644 index b0c4bcb..0000000 --- a/pkgs/toolchain/expect/patches/expect-5.43.0-tcl8.5.patch +++ /dev/null @@ -1,22 +0,0 @@ -diff -up expect-5.43/Dbg.c.tcl8.5 expect-5.43/Dbg.c ---- expect-5.43/Dbg.c.tcl8.5 2002-03-23 05:11:54.000000000 +0100 -+++ expect-5.43/Dbg.c 2008-09-25 12:25:40.000000000 +0200 -@@ -10,6 +10,7 @@ would appreciate credit if this program - - #include <stdio.h> - -+#include "expect_cf.h" - #include "tcldbgcf.h" - #if 0 - /* tclInt.h drags in stdlib. By claiming no-stdlib, force it to drag in */ -diff -up expect-5.43/exp_inter.c.tcl8.5 expect-5.43/exp_inter.c ---- expect-5.43/exp_inter.c.tcl8.5 2004-08-18 04:03:00.000000000 +0200 -+++ expect-5.43/exp_inter.c 2008-09-25 12:25:40.000000000 +0200 -@@ -36,6 +36,7 @@ would appreciate credit if this program - #include <ctype.h> - - #include "tcl.h" -+#include "tclInt.h" - #include "string.h" - - #include "exp_tty_in.h" diff --git a/pkgs/toolchain/expect/patches/expect-5.43.0-tcl_8.5.8_fix-1.patch b/pkgs/toolchain/expect/patches/expect-5.43.0-tcl_8.5.8_fix-1.patch new file mode 100644 index 0000000..5b9596a --- /dev/null +++ b/pkgs/toolchain/expect/patches/expect-5.43.0-tcl_8.5.8_fix-1.patch @@ -0,0 +1,125 @@ +Submitted By: DJ Lucas <dj_AT_linuxfromscratch_DOT_org> +Date: 2008-09-28 +Initial Package Version: 5.43 +Upstream status: Not Submitted +Origin: LFS ticket 2126 (http://wiki.linuxfromscratch.org/lfs/ticket/2126) + Bryan Kadzban <bryan(a)kadzban.is-a-geek.net> +Description: Removes references to functions that Tcl 8.5 no longer exposes + and correct other minor build problems WRT TCL-8.5.x. + +diff -Naur expect-5.43-orig/Dbg.c expect-5.43/Dbg.c +--- expect-5.43-orig/Dbg.c 2002-03-22 22:11:54.000000000 -0600 ++++ expect-5.43/Dbg.c 2008-08-24 01:30:15.000000000 -0500 +@@ -9,7 +9,7 @@ + */ + + #include <stdio.h> +- ++#include "expect_cf.h" + #include "tcldbgcf.h" + #if 0 + /* tclInt.h drags in stdlib. By claiming no-stdlib, force it to drag in */ +diff -Naur expect-5.43-orig/exp_inter.c expect-5.43/exp_inter.c +--- expect-5.43-orig/exp_inter.c 2004-08-17 21:03:00.000000000 -0500 ++++ expect-5.43/exp_inter.c 2008-08-24 01:28:59.000000000 -0500 +@@ -36,6 +36,7 @@ + #include <ctype.h> + + #include "tcl.h" ++#include "tclInt.h" + #include "string.h" + + #include "exp_tty_in.h" +diff -Naur expect-5.43/exp_command.c expect-5.43-patched/exp_command.c +--- expect-5.43/exp_command.c 2004-08-20 13:18:01.000000000 -0400 ++++ expect-5.43-patched/exp_command.c 2008-01-12 11:42:45.000000000 -0500 +@@ -2265,6 +2265,8 @@ + /*NOTREACHED*/ + } + ++static struct exp_cmd_data cmd_data[]; ++ + /*ARGSUSED*/ + static int + Exp_CloseObjCmd(clientData, interp, objc, objv) +@@ -2311,12 +2313,23 @@ + /* Historical note: we used "close" long before there was a */ + /* Tcl builtin by the same name. */ + ++ /* The code that registered this function as the handler for */ ++ /* the "close" command stored away the old handler in the */ ++ /* exp_cmd_data for the "close" command. */ ++ ++ struct exp_cmd_data *cmd_ptr; + Tcl_CmdInfo info; ++ ++ for(cmd_ptr = &cmd_data[0]; cmd_ptr->name; cmd_ptr++) { ++ if(strncmp(cmd_ptr->name, "close", 5) == 0) ++ break; ++ } ++ + Tcl_ResetResult(interp); + if (0 == Tcl_GetCommandInfo(interp,"close",&info)) { + info.clientData = 0; + } +- return(Tcl_CloseObjCmd(info.clientData,interp,objc_orig,objv_orig)); ++ return(cmd_ptr->old_objProc(info.clientData,interp,objc_orig,objv_orig)); + } + + if (chanName) { +@@ -2961,7 +2974,10 @@ + /* if successful (i.e., TCL_RETURN is returned) */ + /* modify the result, so that we will handle it specially */ + +- int result = Tcl_ReturnObjCmd(clientData,interp,objc,objv); ++ Tcl_CmdInfo info; ++ Tcl_GetCommandInfo(interp, "return", &info); ++ ++ int result = info.objProc(clientData,interp,objc,objv); + if (result == TCL_RETURN) + result = EXP_TCL_RETURN; + return result; +@@ -3062,8 +3078,7 @@ + + for (;c->name;c++) { + /* if already defined, don't redefine */ +- if ((c->flags & EXP_REDEFINE) || +- !(Tcl_FindHashEntry(&globalNsPtr->cmdTable,c->name) || ++ if (!(Tcl_FindHashEntry(&globalNsPtr->cmdTable,c->name) || + Tcl_FindHashEntry(&currNsPtr->cmdTable,c->name))) { + if (c->objproc) + Tcl_CreateObjCommand(interp,c->name, +@@ -3072,6 +3087,21 @@ + Tcl_CreateCommand(interp,c->name,c->proc, + c->data,exp_deleteProc); + } ++ else if (c->flags & EXP_REDEFINE) { /* unless the REDEFINE flag is present */ ++ Tcl_CmdInfo info; ++ ++ if (Tcl_GetCommandInfo(interp, c->name, &info)) { ++ c->old_proc = info.proc; ++ c->old_objProc = info.objProc; ++ } ++ ++ if (c->objproc) ++ Tcl_CreateObjCommand(interp,c->name, ++ c->objproc,c->data,exp_deleteObjProc); ++ else ++ Tcl_CreateCommand(interp,c->name,c->proc, ++ c->data,exp_deleteProc); ++ } + if (!(c->name[0] == 'e' && + c->name[1] == 'x' && + c->name[2] == 'p') +diff -Naur expect-5.43/exp_command.h expect-5.43-patched/exp_command.h +--- expect-5.43/exp_command.h 2008-01-12 11:44:11.000000000 -0500 ++++ expect-5.43-patched/exp_command.h 2008-01-12 11:26:05.000000000 -0500 +@@ -297,6 +297,8 @@ + Tcl_CmdProc *proc; + ClientData data; + int flags; ++ Tcl_CmdProc *old_proc; /* these store the procedure for the old command, */ ++ Tcl_ObjCmdProc *old_objProc; /* if any */ + }; + + EXTERN void exp_create_commands _ANSI_ARGS_((Tcl_Interp *, hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. 58a60fde7432a8276ec821019d401fa6e0e46dc4
by git＠ipfire.org 19 Mar '10

19 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via 58a60fde7432a8276ec821019d401fa6e0e46dc4 (commit) via 4732029a723f69870eee80886cb04bb6d00f671e (commit) from e4a79d86036678baa00f1d93273cde035e9e158a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 58a60fde7432a8276ec821019d401fa6e0e46dc4 Merge: 4732029a723f69870eee80886cb04bb6d00f671e e4a79d86036678baa00f1d93273cde035e9e158a Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 15:24:02 2010 +0100 Merge branch 'master' of ssh://git.ipfire.org/pub/git/ipfire-3.x commit 4732029a723f69870eee80886cb04bb6d00f671e Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 14:50:42 2010 +0100 naoki: Do not log everything by default. ----------------------------------------------------------------------- Summary of changes: naoki/util.py | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) Difference in files: diff --git a/naoki/util.py b/naoki/util.py index a514a8a..5898727 100644 --- a/naoki/util.py +++ b/naoki/util.py @@ -126,8 +126,7 @@ def condPersonality(per=None): if res == -1: raise OSError(_errno.value, os.strerror(_errno.value)) -def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True, returnOutput=0, personality=None, *args, **kargs): - logger = kargs.get("logger", getLog()) +def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True, returnOutput=0, personality=None, logger=None, *args, **kargs): output = "" start = time.time() env = kargs.get("env", None) @@ -138,7 +137,8 @@ def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True try: child = None - logger.debug("Executing command: %s" % command) + if logger: + logger.debug("Executing command: %s" % command) child = subprocess.Popen( command, shell=shell, @@ -149,7 +149,7 @@ def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True preexec_fn = preexec, env=env ) - + # use select() to poll for output so we dont block output = logOutput([child.stdout, child.stderr], logger, returnOutput, start, timeout) @@ -178,7 +178,8 @@ def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True if not niceExit: raise commandTimeoutExpired, ("Timeout(%s) expired for command:\n # %s\n%s" % (timeout, command, output)) - logger.debug("Child returncode was: %s" % str(child.returncode)) + if logger: + logger.debug("Child returncode was: %s" % str(child.returncode)) if raiseExc and child.returncode: if returnOutput: raise Error, ("Command failed: \n # %s\n%s" % (command, output), child.returncode) hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. e4a79d86036678baa00f1d93273cde035e9e158a
by git＠ipfire.org 19 Mar '10

19 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via e4a79d86036678baa00f1d93273cde035e9e158a (commit) via fce85abf5846d30f4dcd61a70b3e15141d24ec2c (commit) from 25cea7e2e58e6584a794b4a6c17cd52738140118 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e4a79d86036678baa00f1d93273cde035e9e158a Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 15:18:04 2010 +0100 binutils: Update to 2.20.1. commit fce85abf5846d30f4dcd61a70b3e15141d24ec2c Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 14:42:25 2010 +0100 ccache: Update to 3.0pre0. ----------------------------------------------------------------------- Summary of changes: pkgs/core/binutils/binutils.nm | 2 +- pkgs/core/ccache/ccache.nm | 10 ++++++---- pkgs/core/ccache/patches/ccache-2.4.patch | 12 ------------ 3 files changed, 7 insertions(+), 17 deletions(-) delete mode 100644 pkgs/core/ccache/patches/ccache-2.4.patch Difference in files: diff --git a/pkgs/core/binutils/binutils.nm b/pkgs/core/binutils/binutils.nm index a970694..8cb818f 100644 --- a/pkgs/core/binutils/binutils.nm +++ b/pkgs/core/binutils/binutils.nm @@ -25,7 +25,7 @@ include $(PKGROOT)/Include PKG_NAME = binutils -PKG_VER = 2.20 +PKG_VER = 2.20.1 PKG_REL = 0 PKG_MAINTAINER = Michael Tremer <michael.tremer(a)ipfire.org> diff --git a/pkgs/core/ccache/ccache.nm b/pkgs/core/ccache/ccache.nm index cce011b..63a30b8 100644 --- a/pkgs/core/ccache/ccache.nm +++ b/pkgs/core/ccache/ccache.nm @@ -25,7 +25,7 @@ include $(PKGROOT)/Include PKG_NAME = ccache -PKG_VER = 2.4 +PKG_VER = 3.0pre0 PKG_REL = 0 PKG_MAINTAINER = @@ -34,6 +34,8 @@ PKG_URL = http://ccache.samba.org/ PKG_LICENSE = GPLv2+ PKG_SUMMARY = C/C++ compiler cache. +PKG_DEPS += zlib + define PKG_DESCRIPTION ccache is a compiler cache. It acts as a caching pre-processor to \ C/C++ compilers, using the -E compiler switch and a hash to detect \ @@ -45,9 +47,9 @@ PKG_TARBALL = $(THISAPP).tar.gz CONFIGURE_OPTIONS += --mandir=/usr/share -############################################################################### -# Installation Details -############################################################################### +define STAGE_PREPARE_CMDS + cd $(DIR_APP) && sed -e "s/hash_int(&hash, st.st_mtime);//g" -i ccache.c +endef define STAGE_INSTALL cd $(DIR_APP) && make install DESTDIR=$(BUILDROOT) diff --git a/pkgs/core/ccache/patches/ccache-2.4.patch b/pkgs/core/ccache/patches/ccache-2.4.patch deleted file mode 100644 index b2193c3..0000000 --- a/pkgs/core/ccache/patches/ccache-2.4.patch +++ /dev/null @@ -1,12 +0,0 @@ ---- a/ccache.c 2004-04-15 13:49:38.000000000 +0100 -+++ b/ccache.c 2004-04-15 17:47:59.000000000 +0100 -@@ -322,7 +322,9 @@ - failed(); - } - hash_int(st.st_size); -+#if 0 /* don't hash on the modification time */ - hash_int(st.st_mtime); -+#endif - - /* possibly hash the current working directory */ - if (getenv("CCACHE_HASHDIR")) { hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. 25cea7e2e58e6584a794b4a6c17cd52738140118
by git＠ipfire.org 19 Mar '10

19 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via 25cea7e2e58e6584a794b4a6c17cd52738140118 (commit) from 865a945a4b5895c08b3a7aba885b7f7462e87c01 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 25cea7e2e58e6584a794b4a6c17cd52738140118 Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 14:27:10 2010 +0100 naoki: Fix include of "handlers.py". ----------------------------------------------------------------------- Summary of changes: naoki/__init__.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) Difference in files: diff --git a/naoki/__init__.py b/naoki/__init__.py index 515a553..039eff6 100644 --- a/naoki/__init__.py +++ b/naoki/__init__.py @@ -11,7 +11,6 @@ import terminal import util from constants import * -from handlers import * class Naoki(object): def __init__(self): hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. 865a945a4b5895c08b3a7aba885b7f7462e87c01
by git＠ipfire.org 19 Mar '10

19 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via 865a945a4b5895c08b3a7aba885b7f7462e87c01 (commit) via fdc7c3044fea2e94070e5e868ce99b2bbf1cfdae (commit) via 026c8fcf625113be793f1fd1098009044a53fc25 (commit) via 928cd1193dfaa4eeb6f7bb57dc6fb57352a51ee7 (commit) via 0ec7d269b764d2935aec4a3f62600392f760718f (commit) from a00d6e8e1ba50103623c660a174e9878c92f1542 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 865a945a4b5895c08b3a7aba885b7f7462e87c01 Merge: fdc7c3044fea2e94070e5e868ce99b2bbf1cfdae a00d6e8e1ba50103623c660a174e9878c92f1542 Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 14:22:50 2010 +0100 Merge branch 'master' of ssh://git.ipfire.org/pub/git/ipfire-3.x commit fdc7c3044fea2e94070e5e868ce99b2bbf1cfdae Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 14:22:30 2010 +0100 naoki: Fix logging. commit 026c8fcf625113be793f1fd1098009044a53fc25 Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 13:30:02 2010 +0100 naoki: Remove argparse module. commit 928cd1193dfaa4eeb6f7bb57dc6fb57352a51ee7 Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Fri Mar 19 13:29:19 2010 +0100 naoki: Refactor some more things on naoki. commit 0ec7d269b764d2935aec4a3f62600392f760718f Author: Michael Tremer <michael.tremer(a)ipfire.org> Date: Thu Mar 18 23:06:14 2010 +0100 naoki: Refactoring of command line parsing. ----------------------------------------------------------------------- Summary of changes: make.sh | 124 +--- naoki/__init__.py | 240 ++++-- naoki/argparse.py | 2270 ----------------------------------------------------- naoki/backend.py | 319 ++++++++ naoki/chroot.py | 46 +- naoki/logger.py | 71 ++- naoki/terminal.py | 256 ++++++ naoki/util.py | 2 +- 8 files changed, 818 insertions(+), 2510 deletions(-) delete mode 100644 naoki/argparse.py create mode 100644 naoki/backend.py Difference in files: diff --git a/make.sh b/make.sh index c8c1823..46b3ae0 100755 --- a/make.sh +++ b/make.sh @@ -1,16 +1,8 @@ #!/usr/bin/python import sys -try: - import argparse -except ImportError: - import naoki.argparse as argparse - import naoki -arches = naoki.arches -config = naoki.config - # silence Python 2.6 buggy warnings about Exception.message if sys.version_info[:2] == (2, 6): import warnings @@ -19,120 +11,12 @@ if sys.version_info[:2] == (2, 6): message="BaseException.message has been deprecated as of Python 2.6", category=DeprecationWarning) -# Command line parsing -parser = argparse.ArgumentParser( - description = "Command to control the naoki buildsystem" -) - -parser.add_argument("-q", "--quiet", action="store_true", help="run in silent mode") -parser.add_argument("-d", "--debug", action="store_true", help="run in debug mode") -parser.add_argument("-a", "--arch", default=arches.default["name"], help="set architecture") -subparsers = parser.add_subparsers(help="sub-command help") - -# Build command -parser_build = subparsers.add_parser("build", help="build command") -parser_build.set_defaults(action="build") -parser_build.add_argument("package", nargs="+", help="package name") - - -# Toolchain command -parser_toolchain = subparsers.add_parser("toolchain", help="toolchain command") -parser_toolchain.set_defaults(action="toolchain") -subparsers_toolchain = parser_toolchain.add_subparsers(help="sub-command help") - - # toolchain build -parser_toolchain_build = subparsers_toolchain.add_parser("build", help="build toolchain") -parser_toolchain_build.set_defaults(subaction="build") - - # toolchain build -parser_toolchain_download = subparsers_toolchain.add_parser("download", help="download toolchain") -parser_toolchain_download.set_defaults(subaction="download") - - -# Package command -parser_package = subparsers.add_parser("package", help="package command") -parser_package.set_defaults(action="package") -subparsers_package = parser_package.add_subparsers(help="sub-command help") - - # package info [-l, --long] -parser_package_info = subparsers_package.add_parser("info", help="show package information") -parser_package_info.set_defaults(subaction="info") -parser_package_info.add_argument("-l", "--long", action="store_true", help="print in long format") -parser_package_info.add_argument("--machine", action="store_true", help="output in machine readable format") -parser_package_info.add_argument("--wiki", action="store_true", help="output in wiki format") -parser_package_info.add_argument("package", nargs="+", help="package name") - - # package tree -parser_package_tree = subparsers_package.add_parser("tree", help="show package tree") -parser_package_tree.set_defaults(subaction="tree") - - # package list [-l, --long] -parser_package_list = subparsers_package.add_parser("list", help="show package list") -parser_package_list.set_defaults(subaction="list") -parser_package_list.add_argument("-l", "--long", action="store_true", help="print list in long format") - - # package groups -parser_package_groups = subparsers_package.add_parser("groups", help="show package groups") -parser_package_groups.set_defaults(subaction="groups") -parser_package_groups.add_argument("--wiki", action="store_true", help="output in wiki format") - - -# Source command -parser_source = subparsers.add_parser("source", help="source command") -parser_source.set_defaults(action="source") -subparsers_source = parser_source.add_subparsers(help="sub-command help") - - # source download -parser_source_download = subparsers_source.add_parser("download", help="download source tarballs") -parser_source_download.set_defaults(subaction="download") -parser_source_download.add_argument("package", nargs="*", help="package name") - - # source upload -parser_source_upload = subparsers_source.add_parser("upload", help="upload source tarballs") -parser_source_upload.set_defaults(subaction="upload") -parser_source_upload.add_argument("package", nargs="*", help="package name") - - # source clean -parser_source_clean = subparsers_source.add_parser("clean", help="cleanup unused source tarballs") -parser_source_clean.set_defaults(subaction="clean") - - -# Check command -parser_check = subparsers.add_parser("check", help="check command") -parser_check.set_defaults(action="check") -subparsers_check = parser_check.add_subparsers(help="sub-command help") - - # check host -parser_check_host = subparsers_check.add_parser("host", help="check if host fulfills requirements") -parser_check_host.set_defaults(subaction="host") - - -# Batch command -parser_batch = subparsers.add_parser("batch", help="batch command") -parser_batch.set_defaults(action="batch") -subparsers_batch = parser_batch.add_subparsers(help="sub-command help") - - # batch cron -parser_batch_cron = subparsers_batch.add_parser("cron", help="gets called by a cron daemon") -parser_batch_cron.set_defaults(subaction="cron") - - -# parse the command line -args = parser.parse_args() - -if args.debug: - print "Command line arguments:", args - -# Set default arch -arches.set(args.arch) - -kwargs = {} -for key, val in args._get_kwargs(): - kwargs[key] = val +# Initialize system +n = naoki.Naoki() try: - n = naoki.Naoki() - n(**kwargs) + # Run... + n.run() exitStatus = 0 except (SystemExit,): diff --git a/naoki/__init__.py b/naoki/__init__.py index 212f81f..515a553 100644 --- a/naoki/__init__.py +++ b/naoki/__init__.py @@ -1,121 +1,195 @@ #!/usr/bin/python import ConfigParser -import curses -import logging -import logging.config -import logging.handlers import os.path import sys import time +import backend import logger -import package +import terminal import util from constants import * - -# fix for python 2.4 logging module bug: -logging.raiseExceptions = 0 +from handlers import * class Naoki(object): def __init__(self): - self.setup_logging() + # First, setup the logging + self.logging = logger.Logging(self) + + # Second, parse the command line options + self.cli = terminal.Commandline(self) self.log.debug("Successfully initialized naoki instance") for k, v in config.items(): self.log.debug(" %s: %s" % (k, v)) - def setup_logging(self): - self.log = logging.getLogger() + def run(self): + args = self.cli.args - log_ini = config["log_config_file"] - if os.path.exists(log_ini): - logging.config.fileConfig(log_ini) + # If there is no action provided, exit + if not args.has_key("action"): + self.cli.help() + sys.exit(1) - if sys.stderr.isatty(): - curses.setupterm() - self.log.handlers[0].setFormatter(logger._ColorLogFormatter()) + actionmap = { + "build" : self.call_build, + "toolchain" : self.call_toolchain, + "package" : self.call_package, + "source" : self.call_source, + } - if config["quiet"]: - self.log.handlers[0].setLevel(logging.WARNING) - else: - self.log.handlers[0].setLevel(logging.INFO) - - if not os.path.isdir(LOGDIR): - os.makedirs(LOGDIR) - fh = logging.handlers.RotatingFileHandler(config["log_file"], - maxBytes=10*1024**2, backupCount=6) - fh.setFormatter(logging.Formatter("[%(levelname)s] %(message)s")) - fh.setLevel(logging.NOTSET) - self.log.addHandler(fh) - - def __call__(self, action, **kwargs): - if action == "build": - self.call_build(kwargs.get("package")) - - elif action == "toolchain": - self.call_toolchain(kwargs.get("subaction"), kwargs.get("arch")) - - elif action == "package": - self.call_package(kwargs.pop("subaction"), **kwargs) + return actionmap[args.action.name](args.action) - def call_toolchain(self, subaction, arch): - tc = chroot.Toolchain(arch) - - if subaction == "build": - tc.build() + def call_toolchain(self, args): + if not args.has_key("action"): + self.cli.help() + sys.exit(1) + + actionmap = { + "build" : self.call_toolchain_build, + "download" : self.call_toolchain_download, + } + + return actionmap[args.action.name](args.action) + + def call_toolchain_build(self, args): + toolchain = chroot.Toolchain(arch) + + return toolchain.build() + + def call_toolchain_download(self, args): + toolchain = chroot.Toolchain(arch) - elif subaction == "download": - tc.download() + return toolchain.download() - def call_build(self, packages): + def call_build(self, args): force = True - if packages == ["all"]: + if args.packages == ["all"]: force = False - packages = package.list() + package_names = backend.get_package_names() else: - packages = [package.find(p) for p in packages] - for p in packages: - if not p: packages.remove(p) + package_names = args.packages + + packages = [] + for package in package_names: + package = backend.Package(package, naoki=self) + packages.append(package) + + if len(packages) >= 2: + packages_sorted = backend.depsort(packages) + if packages_sorted == packages: + self.log.warn("Packages were resorted for build: %s" % packages_sorted) + packages = packages_sorted + + for package in packages: + environ = chroot.Environment(package) + + if not environ.toolchain.exists: + self.log.error("You need to build or download a toolchain first.") + continue - self._build(packages, force=force) + environ.build() + + def call_package(self, args): + if not args.has_key("action"): + self.cli.help() + sys.exit(1) + + actionmap = { + "info" : self.call_package_info, + "list" : self.call_package_list, + "tree" : self.call_package_tree, + "groups" : self.call_package_groups, + } + + return actionmap[args.action.name](args.action) + + def call_package_info(self, args): + packages = args.packages or backend.get_package_names() + + for package in packages: + package = backend.PackageInfo(package) + if args.long: + print package.fmtstr("""\ +-------------------------------------------------------------------------------- +Name : %(name)s +Version : %(version)s +Release : %(release)s + + %(summary)s + +%(description)s + +Maintainer : %(maintainer)s +License : %(license)s + +Files : %(objects)s +Patches : %(patches)s +--------------------------------------------------------------------------------\ +""") + else: + print package.fmtstr("""\ +-------------------------------------------------------------------------------- +Name : %(name)s +Version : %(version)s +Release : %(release)s + + %(summary)s + +--------------------------------------------------------------------------------\ +""") + + def call_package_list(self, args): + for package in self.package_names: + package = backend.PackageInfo(package) + if args.long: + print package.fmtstr("%(name)-32s | %(version)-15s | %(summary)s") + else: + print package.fmtstr("%(name)s") + + def call_package_tree(self, args): + print "TBD" + + def call_package_groups(self, args): + groups = backend.get_group_names() + if args.wiki: + print "====== All available groups of packages ======" + for group in groups: + print "===== %s =====" % group + for package in backend.get_package_names(): + package = backend.PackageInfo(package) + if not package.group == group: + continue + + print package.fmtstr(" * [[.package:%(name)s|%(name)s]] - %(summary)s") - def call_package(self, subaction, **kwargs): - if subaction == "list": - for pkg in self.packages: - print pkg.info_line(long=kwargs["long"]) + else: + print "\n".join(groups) - elif subaction == "info": - packages = [package.find(pkg) for pkg in kwargs.get("package")] - packages.sort() + def call_source(self, args): + if not args.has_key("action"): + self.cli.help() + sys.exit(1) - if kwargs["wiki"]: - for pkg in packages: - print pkg.info_wiki() - return - - delimiter = "----------------------------------------------------\n" + actionmap = { + "download" : self.call_source_download, + "upload" : self.call_source_upload, + } - print delimiter.join([pkg.info(long=kwargs["long"]) for pkg in packages]) - - elif subaction == "tree": - print package.deptree(self.packages) - - elif subaction == "groups": - groups = package.groups() + return actionmap[args.action.name](args.action) - if kwargs["wiki"]: - print "====== All available groups of packages ======" - for group in groups: - print group.wiki_headline() - for pkg in group.packages: - print pkg.info_wiki(long=False) + def call_source_download(self, args): + packages = args.packages or backend.get_package_names() - return + for package in packages: + package = backend.Package(package, naoki=self) + package.download() - print "\n".join(package.group_names()) + def call_source_upload(self, args): + pass # TODO def _build(self, packages, force=False): requeue = [] @@ -147,5 +221,5 @@ class Naoki(object): build.build() @property - def packages(self): - return package.list() + def package_names(self): + return backend.get_package_names() diff --git a/naoki/argparse.py b/naoki/argparse.py deleted file mode 100644 index ce1adb9..0000000 --- a/naoki/argparse.py +++ /dev/null @@ -1,2270 +0,0 @@ -# -*- coding: utf-8 -*- - -# Copyright © 2006-2009 Steven J. Bethard <steven.bethard(a)gmail.com>. -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may not -# use this file except in compliance with the License. You may obtain a copy -# of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. - -"""Command-line parsing library - -This module is an optparse-inspired command-line parsing library that: - - - handles both optional and positional arguments - - produces highly informative usage messages - - supports parsers that dispatch to sub-parsers - -The following is a simple usage example that sums integers from the -command-line and writes the result to a file:: - - parser = argparse.ArgumentParser( - description='sum the integers at the command line') - parser.add_argument( - 'integers', metavar='int', nargs='+', type=int, - help='an integer to be summed') - parser.add_argument( - '--log', default=sys.stdout, type=argparse.FileType('w'), - help='the file where the sum should be written') - args = parser.parse_args() - args.log.write('%s' % sum(args.integers)) - args.log.close() - -The module contains the following public classes: - - - ArgumentParser -- The main entry point for command-line parsing. As the - example above shows, the add_argument() method is used to populate - the parser with actions for optional and positional arguments. Then - the parse_args() method is invoked to convert the args at the - command-line into an object with attributes. - - - ArgumentError -- The exception raised by ArgumentParser objects when - there are errors with the parser's actions. Errors raised while - parsing the command-line are caught by ArgumentParser and emitted - as command-line messages. - - - FileType -- A factory for defining types of files to be created. As the - example above shows, instances of FileType are typically passed as - the type= argument of add_argument() calls. - - - Action -- The base class for parser actions. Typically actions are - selected by passing strings like 'store_true' or 'append_const' to - the action= argument of add_argument(). However, for greater - customization of ArgumentParser actions, subclasses of Action may - be defined and passed as the action= argument. - - - HelpFormatter, RawDescriptionHelpFormatter, RawTextHelpFormatter, - ArgumentDefaultsHelpFormatter -- Formatter classes which - may be passed as the formatter_class= argument to the - ArgumentParser constructor. HelpFormatter is the default, - RawDescriptionHelpFormatter and RawTextHelpFormatter tell the parser - not to change the formatting for help text, and - ArgumentDefaultsHelpFormatter adds information about argument defaults - to the help. - -All other classes in this module are considered implementation details. -(Also note that HelpFormatter and RawDescriptionHelpFormatter are only -considered public as object names -- the API of the formatter objects is -still considered an implementation detail.) -""" - -__version__ = '1.0.1' -__all__ = [ - 'ArgumentParser', - 'ArgumentError', - 'Namespace', - 'Action', - 'FileType', - 'HelpFormatter', - 'RawDescriptionHelpFormatter', - 'RawTextHelpFormatter' - 'ArgumentDefaultsHelpFormatter', -] - - -import copy as _copy -import os as _os -import re as _re -import sys as _sys -import textwrap as _textwrap - -from gettext import gettext as _ - -try: - _set = set -except NameError: - from sets import Set as _set - -try: - _basestring = basestring -except NameError: - _basestring = str - -try: - _sorted = sorted -except NameError: - - def _sorted(iterable, reverse=False): - result = list(iterable) - result.sort() - if reverse: - result.reverse() - return result - -# silence Python 2.6 buggy warnings about Exception.message -if _sys.version_info[:2] == (2, 6): - import warnings - warnings.filterwarnings( - action='ignore', - message='BaseException.message has been deprecated as of Python 2.6', - category=DeprecationWarning, - module='argparse') - - -SUPPRESS = '==SUPPRESS==' - -OPTIONAL = '?' -ZERO_OR_MORE = '*' -ONE_OR_MORE = '+' -PARSER = '==PARSER==' - -# ============================= -# Utility functions and classes -# ============================= - -class _AttributeHolder(object): - """Abstract base class that provides __repr__. - - The __repr__ method returns a string in the format:: - ClassName(attr=name, attr=name, ...) - The attributes are determined either by a class-level attribute, - '_kwarg_names', or by inspecting the instance __dict__. - """ - - def __repr__(self): - type_name = type(self).__name__ - arg_strings = [] - for arg in self._get_args(): - arg_strings.append(repr(arg)) - for name, value in self._get_kwargs(): - arg_strings.append('%s=%r' % (name, value)) - return '%s(%s)' % (type_name, ', '.join(arg_strings)) - - def _get_kwargs(self): - return _sorted(self.__dict__.items()) - - def _get_args(self): - return [] - - -def _ensure_value(namespace, name, value): - if getattr(namespace, name, None) is None: - setattr(namespace, name, value) - return getattr(namespace, name) - - -# =============== -# Formatting Help -# =============== - -class HelpFormatter(object): - """Formatter for generating usage messages and argument help strings. - - Only the name of this class is considered a public API. All the methods - provided by the class are considered an implementation detail. - """ - - def __init__(self, - prog, - indent_increment=2, - max_help_position=24, - width=None): - - # default setting for width - if width is None: - try: - width = int(_os.environ['COLUMNS']) - except (KeyError, ValueError): - width = 80 - width -= 2 - - self._prog = prog - self._indent_increment = indent_increment - self._max_help_position = max_help_position - self._width = width - - self._current_indent = 0 - self._level = 0 - self._action_max_length = 0 - - self._root_section = self._Section(self, None) - self._current_section = self._root_section - - self._whitespace_matcher = _re.compile(r'\s+') - self._long_break_matcher = _re.compile(r'\n\n\n+') - - # =============================== - # Section and indentation methods - # =============================== - def _indent(self): - self._current_indent += self._indent_increment - self._level += 1 - - def _dedent(self): - self._current_indent -= self._indent_increment - assert self._current_indent >= 0, 'Indent decreased below 0.' - self._level -= 1 - - class _Section(object): - - def __init__(self, formatter, parent, heading=None): - self.formatter = formatter - self.parent = parent - self.heading = heading - self.items = [] - - def format_help(self): - # format the indented section - if self.parent is not None: - self.formatter._indent() - join = self.formatter._join_parts - for func, args in self.items: - func(*args) - item_help = join([func(*args) for func, args in self.items]) - if self.parent is not None: - self.formatter._dedent() - - # return nothing if the section was empty - if not item_help: - return '' - - # add the heading if the section was non-empty - if self.heading is not SUPPRESS and self.heading is not None: - current_indent = self.formatter._current_indent - heading = '%*s%s:\n' % (current_indent, '', self.heading) - else: - heading = '' - - # join the section-initial newline, the heading and the help - return join(['\n', heading, item_help, '\n']) - - def _add_item(self, func, args): - self._current_section.items.append((func, args)) - - # ======================== - # Message building methods - # ======================== - def start_section(self, heading): - self._indent() - section = self._Section(self, self._current_section, heading) - self._add_item(section.format_help, []) - self._current_section = section - - def end_section(self): - self._current_section = self._current_section.parent - self._dedent() - - def add_text(self, text): - if text is not SUPPRESS and text is not None: - self._add_item(self._format_text, [text]) - - def add_usage(self, usage, actions, groups, prefix=None): - if usage is not SUPPRESS: - args = usage, actions, groups, prefix - self._add_item(self._format_usage, args) - - def add_argument(self, action): - if action.help is not SUPPRESS: - - # find all invocations - get_invocation = self._format_action_invocation - invocations = [get_invocation(action)] - for subaction in self._iter_indented_subactions(action): - invocations.append(get_invocation(subaction)) - - # update the maximum item length - invocation_length = max([len(s) for s in invocations]) - action_length = invocation_length + self._current_indent - self._action_max_length = max(self._action_max_length, - action_length) - - # add the item to the list - self._add_item(self._format_action, [action]) - - def add_arguments(self, actions): - for action in actions: - self.add_argument(action) - - # ======================= - # Help-formatting methods - # ======================= - def format_help(self): - help = self._root_section.format_help() - if help: - help = self._long_break_matcher.sub('\n\n', help) - help = help.strip('\n') + '\n' - return help - - def _join_parts(self, part_strings): - return ''.join([part - for part in part_strings - if part and part is not SUPPRESS]) - - def _format_usage(self, usage, actions, groups, prefix): - if prefix is None: - prefix = _('usage: ') - - # if usage is specified, use that - if usage is not None: - usage = usage % dict(prog=self._prog) - - # if no optionals or positionals are available, usage is just prog - elif usage is None and not actions: - usage = '%(prog)s' % dict(prog=self._prog) - - # if optionals and positionals are available, calculate usage - elif usage is None: - prog = '%(prog)s' % dict(prog=self._prog) - - # split optionals from positionals - optionals = [] - positionals = [] - for action in actions: - if action.option_strings: - optionals.append(action) - else: - positionals.append(action) - - # build full usage string - format = self._format_actions_usage - action_usage = format(optionals + positionals, groups) - usage = ' '.join([s for s in [prog, action_usage] if s]) - - # wrap the usage parts if it's too long - text_width = self._width - self._current_indent - if len(prefix) + len(usage) > text_width: - - # break usage into wrappable parts - part_regexp = r'$.*?$+|\[.*?\]+|\S+' - opt_usage = format(optionals, groups) - pos_usage = format(positionals, groups) - opt_parts = _re.findall(part_regexp, opt_usage) - pos_parts = _re.findall(part_regexp, pos_usage) - assert ' '.join(opt_parts) == opt_usage - assert ' '.join(pos_parts) == pos_usage - - # helper for wrapping lines - def get_lines(parts, indent, prefix=None): - lines = [] - line = [] - if prefix is not None: - line_len = len(prefix) - 1 - else: - line_len = len(indent) - 1 - for part in parts: - if line_len + 1 + len(part) > text_width: - lines.append(indent + ' '.join(line)) - line = [] - line_len = len(indent) - 1 - line.append(part) - line_len += len(part) + 1 - if line: - lines.append(indent + ' '.join(line)) - if prefix is not None: - lines[0] = lines[0][len(indent):] - return lines - - # if prog is short, follow it with optionals or positionals - if len(prefix) + len(prog) <= 0.75 * text_width: - indent = ' ' * (len(prefix) + len(prog) + 1) - if opt_parts: - lines = get_lines([prog] + opt_parts, indent, prefix) - lines.extend(get_lines(pos_parts, indent)) - elif pos_parts: - lines = get_lines([prog] + pos_parts, indent, prefix) - else: - lines = [prog] - - # if prog is long, put it on its own line - else: - indent = ' ' * len(prefix) - parts = opt_parts + pos_parts - lines = get_lines(parts, indent) - if len(lines) > 1: - lines = [] - lines.extend(get_lines(opt_parts, indent)) - lines.extend(get_lines(pos_parts, indent)) - lines = [prog] + lines - - # join lines into usage - usage = '\n'.join(lines) - - # prefix with 'usage:' - return '%s%s\n\n' % (prefix, usage) - - def _format_actions_usage(self, actions, groups): - # find group indices and identify actions in groups - group_actions = _set() - inserts = {} - for group in groups: - try: - start = actions.index(group._group_actions[0]) - except ValueError: - continue - else: - end = start + len(group._group_actions) - if actions[start:end] == group._group_actions: - for action in group._group_actions: - group_actions.add(action) - if not group.required: - inserts[start] = '[' - inserts[end] = ']' - else: - inserts[start] = '(' - inserts[end] = ')' - for i in range(start + 1, end): - inserts[i] = '|' - - # collect all actions format strings - parts = [] - for i, action in enumerate(actions): - - # suppressed arguments are marked with None - # remove | separators for suppressed arguments - if action.help is SUPPRESS: - parts.append(None) - if inserts.get(i) == '|': - inserts.pop(i) - elif inserts.get(i + 1) == '|': - inserts.pop(i + 1) - - # produce all arg strings - elif not action.option_strings: - part = self._format_args(action, action.dest) - - # if it's in a group, strip the outer [] - if action in group_actions: - if part[0] == '[' and part[-1] == ']': - part = part[1:-1] - - # add the action string to the list - parts.append(part) - - # produce the first way to invoke the option in brackets - else: - option_string = action.option_strings[0] - - # if the Optional doesn't take a value, format is: - # -s or --long - if action.nargs == 0: - part = '%s' % option_string - - # if the Optional takes a value, format is: - # -s ARGS or --long ARGS - else: - default = action.dest.upper() - args_string = self._format_args(action, default) - part = '%s %s' % (option_string, args_string) - - # make it look optional if it's not required or in a group - if not action.required and action not in group_actions: - part = '[%s]' % part - - # add the action string to the list - parts.append(part) - - # insert things at the necessary indices - for i in _sorted(inserts, reverse=True): - parts[i:i] = [inserts[i]] - - # join all the action items with spaces - text = ' '.join([item for item in parts if item is not None]) - - # clean up separators for mutually exclusive groups - open = r'[\[(]' - close = r'[\])]' - text = _re.sub(r'(%s) ' % open, r'\1', text) - text = _re.sub(r' (%s)' % close, r'\1', text) - text = _re.sub(r'%s *%s' % (open, close), r'', text) - text = _re.sub(r'$([^|]*)$', r'\1', text) - text = text.strip() - - # return the text - return text - - def _format_text(self, text): - text_width = self._width - self._current_indent - indent = ' ' * self._current_indent - return self._fill_text(text, text_width, indent) + '\n\n' - - def _format_action(self, action): - # determine the required width and the entry label - help_position = min(self._action_max_length + 2, - self._max_help_position) - help_width = self._width - help_position - action_width = help_position - self._current_indent - 2 - action_header = self._format_action_invocation(action) - - # ho nelp; start on same line and add a final newline - if not action.help: - tup = self._current_indent, '', action_header - action_header = '%*s%s\n' % tup - - # short action name; start on the same line and pad two spaces - elif len(action_header) <= action_width: - tup = self._current_indent, '', action_width, action_header - action_header = '%*s%-*s ' % tup - indent_first = 0 - - # long action name; start on the next line - else: - tup = self._current_indent, '', action_header - action_header = '%*s%s\n' % tup - indent_first = help_position - - # collect the pieces of the action help - parts = [action_header] - - # if there was help for the action, add lines of help text - if action.help: - help_text = self._expand_help(action) - help_lines = self._split_lines(help_text, help_width) - parts.append('%*s%s\n' % (indent_first, '', help_lines[0])) - for line in help_lines[1:]: - parts.append('%*s%s\n' % (help_position, '', line)) - - # or add a newline if the description doesn't end with one - elif not action_header.endswith('\n'): - parts.append('\n') - - # if there are any sub-actions, add their help as well - for subaction in self._iter_indented_subactions(action): - parts.append(self._format_action(subaction)) - - # return a single string - return self._join_parts(parts) - - def _format_action_invocation(self, action): - if not action.option_strings: - metavar, = self._metavar_formatter(action, action.dest)(1) - return metavar - - else: - parts = [] - - # if the Optional doesn't take a value, format is: - # -s, --long - if action.nargs == 0: - parts.extend(action.option_strings) - - # if the Optional takes a value, format is: - # -s ARGS, --long ARGS - else: - default = action.dest.upper() - args_string = self._format_args(action, default) - for option_string in action.option_strings: - parts.append('%s %s' % (option_string, args_string)) - - return ', '.join(parts) - - def _metavar_formatter(self, action, default_metavar): - if action.metavar is not None: - result = action.metavar - elif action.choices is not None: - choice_strs = [str(choice) for choice in action.choices] - result = '{%s}' % ','.join(choice_strs) - else: - result = default_metavar - - def format(tuple_size): - if isinstance(result, tuple): - return result - else: - return (result, ) * tuple_size - return format - - def _format_args(self, action, default_metavar): - get_metavar = self._metavar_formatter(action, default_metavar) - if action.nargs is None: - result = '%s' % get_metavar(1) - elif action.nargs == OPTIONAL: - result = '[%s]' % get_metavar(1) - elif action.nargs == ZERO_OR_MORE: - result = '[%s [%s ...]]' % get_metavar(2) - elif action.nargs == ONE_OR_MORE: - result = '%s [%s ...]' % get_metavar(2) - elif action.nargs is PARSER: - result = '%s ...' % get_metavar(1) - else: - formats = ['%s' for _ in range(action.nargs)] - result = ' '.join(formats) % get_metavar(action.nargs) - return result - - def _expand_help(self, action): - params = dict(vars(action), prog=self._prog) - for name in list(params): - if params[name] is SUPPRESS: - del params[name] - if params.get('choices') is not None: - choices_str = ', '.join([str(c) for c in params['choices']]) - params['choices'] = choices_str - return self._get_help_string(action) % params - - def _iter_indented_subactions(self, action): - try: - get_subactions = action._get_subactions - except AttributeError: - pass - else: - self._indent() - for subaction in get_subactions(): - yield subaction - self._dedent() - - def _split_lines(self, text, width): - text = self._whitespace_matcher.sub(' ', text).strip() - return _textwrap.wrap(text, width) - - def _fill_text(self, text, width, indent): - text = self._whitespace_matcher.sub(' ', text).strip() - return _textwrap.fill(text, width, initial_indent=indent, - subsequent_indent=indent) - - def _get_help_string(self, action): - return action.help - - -class RawDescriptionHelpFormatter(HelpFormatter): - """Help message formatter which retains any formatting in descriptions. - - Only the name of this class is considered a public API. All the methods - provided by the class are considered an implementation detail. - """ - - def _fill_text(self, text, width, indent): - return ''.join([indent + line for line in text.splitlines(True)]) - - -class RawTextHelpFormatter(RawDescriptionHelpFormatter): - """Help message formatter which retains formatting of all help text. - - Only the name of this class is considered a public API. All the methods - provided by the class are considered an implementation detail. - """ - - def _split_lines(self, text, width): - return text.splitlines() - - -class ArgumentDefaultsHelpFormatter(HelpFormatter): - """Help message formatter which adds default values to argument help. - - Only the name of this class is considered a public API. All the methods - provided by the class are considered an implementation detail. - """ - - def _get_help_string(self, action): - help = action.help - if '%(default)' not in action.help: - if action.default is not SUPPRESS: - defaulting_nargs = [OPTIONAL, ZERO_OR_MORE] - if action.option_strings or action.nargs in defaulting_nargs: - help += ' (default: %(default)s)' - return help - - -# ===================== -# Options and Arguments -# ===================== - -def _get_action_name(argument): - if argument is None: - return None - elif argument.option_strings: - return '/'.join(argument.option_strings) - elif argument.metavar not in (None, SUPPRESS): - return argument.metavar - elif argument.dest not in (None, SUPPRESS): - return argument.dest - else: - return None - - -class ArgumentError(Exception): - """An error from creating or using an argument (optional or positional). - - The string value of this exception is the message, augmented with - information about the argument that caused it. - """ - - def __init__(self, argument, message): - self.argument_name = _get_action_name(argument) - self.message = message - - def __str__(self): - if self.argument_name is None: - format = '%(message)s' - else: - format = 'argument %(argument_name)s: %(message)s' - return format % dict(message=self.message, - argument_name=self.argument_name) - -# ============== -# Action classes -# ============== - -class Action(_AttributeHolder): - """Information about how to convert command line strings to Python objects. - - Action objects are used by an ArgumentParser to represent the information - needed to parse a single argument from one or more strings from the - command line. The keyword arguments to the Action constructor are also - all attributes of Action instances. - - Keyword Arguments: - - - option_strings -- A list of command-line option strings which - should be associated with this action. - - - dest -- The name of the attribute to hold the created object(s) - - - nargs -- The number of command-line arguments that should be - consumed. By default, one argument will be consumed and a single - value will be produced. Other values include: - - N (an integer) consumes N arguments (and produces a list) - - '?' consumes zero or one arguments - - '*' consumes zero or more arguments (and produces a list) - - '+' consumes one or more arguments (and produces a list) - Note that the difference between the default and nargs=1 is that - with the default, a single value will be produced, while with - nargs=1, a list containing a single value will be produced. - - - const -- The value to be produced if the option is specified and the - option uses an action that takes no values. - - - default -- The value to be produced if the option is not specified. - - - type -- The type which the command-line arguments should be converted - to, should be one of 'string', 'int', 'float', 'complex' or a - callable object that accepts a single string argument. If None, - 'string' is assumed. - - - choices -- A container of values that should be allowed. If not None, - after a command-line argument has been converted to the appropriate - type, an exception will be raised if it is not a member of this - collection. - - - required -- True if the action must always be specified at the - command line. This is only meaningful for optional command-line - arguments. - - - help -- The help string describing the argument. - - - metavar -- The name to be used for the option's argument with the - help string. If None, the 'dest' value will be used as the name. - """ - - def __init__(self, - option_strings, - dest, - nargs=None, - const=None, - default=None, - type=None, - choices=None, - required=False, - help=None, - metavar=None): - self.option_strings = option_strings - self.dest = dest - self.nargs = nargs - self.const = const - self.default = default - self.type = type - self.choices = choices - self.required = required - self.help = help - self.metavar = metavar - - def _get_kwargs(self): - names = [ - 'option_strings', - 'dest', - 'nargs', - 'const', - 'default', - 'type', - 'choices', - 'help', - 'metavar', - ] - return [(name, getattr(self, name)) for name in names] - - def __call__(self, parser, namespace, values, option_string=None): - raise NotImplementedError(_('.__call__() not defined')) - - -class _StoreAction(Action): - - def __init__(self, - option_strings, - dest, - nargs=None, - const=None, - default=None, - type=None, - choices=None, - required=False, - help=None, - metavar=None): - if nargs == 0: - raise ValueError('nargs for store actions must be > 0; if you ' - 'have nothing to store, actions such as store ' - 'true or store const may be more appropriate') - if const is not None and nargs != OPTIONAL: - raise ValueError('nargs must be %r to supply const' % OPTIONAL) - super(_StoreAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=nargs, - const=const, - default=default, - type=type, - choices=choices, - required=required, - help=help, - metavar=metavar) - - def __call__(self, parser, namespace, values, option_string=None): - setattr(namespace, self.dest, values) - - -class _StoreConstAction(Action): - - def __init__(self, - option_strings, - dest, - const, - default=None, - required=False, - help=None, - metavar=None): - super(_StoreConstAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=0, - const=const, - default=default, - required=required, - help=help) - - def __call__(self, parser, namespace, values, option_string=None): - setattr(namespace, self.dest, self.const) - - -class _StoreTrueAction(_StoreConstAction): - - def __init__(self, - option_strings, - dest, - default=False, - required=False, - help=None): - super(_StoreTrueAction, self).__init__( - option_strings=option_strings, - dest=dest, - const=True, - default=default, - required=required, - help=help) - - -class _StoreFalseAction(_StoreConstAction): - - def __init__(self, - option_strings, - dest, - default=True, - required=False, - help=None): - super(_StoreFalseAction, self).__init__( - option_strings=option_strings, - dest=dest, - const=False, - default=default, - required=required, - help=help) - - -class _AppendAction(Action): - - def __init__(self, - option_strings, - dest, - nargs=None, - const=None, - default=None, - type=None, - choices=None, - required=False, - help=None, - metavar=None): - if nargs == 0: - raise ValueError('nargs for append actions must be > 0; if arg ' - 'strings are not supplying the value to append, ' - 'the append const action may be more appropriate') - if const is not None and nargs != OPTIONAL: - raise ValueError('nargs must be %r to supply const' % OPTIONAL) - super(_AppendAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=nargs, - const=const, - default=default, - type=type, - choices=choices, - required=required, - help=help, - metavar=metavar) - - def __call__(self, parser, namespace, values, option_string=None): - items = _copy.copy(_ensure_value(namespace, self.dest, [])) - items.append(values) - setattr(namespace, self.dest, items) - - -class _AppendConstAction(Action): - - def __init__(self, - option_strings, - dest, - const, - default=None, - required=False, - help=None, - metavar=None): - super(_AppendConstAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=0, - const=const, - default=default, - required=required, - help=help, - metavar=metavar) - - def __call__(self, parser, namespace, values, option_string=None): - items = _copy.copy(_ensure_value(namespace, self.dest, [])) - items.append(self.const) - setattr(namespace, self.dest, items) - - -class _CountAction(Action): - - def __init__(self, - option_strings, - dest, - default=None, - required=False, - help=None): - super(_CountAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=0, - default=default, - required=required, - help=help) - - def __call__(self, parser, namespace, values, option_string=None): - new_count = _ensure_value(namespace, self.dest, 0) + 1 - setattr(namespace, self.dest, new_count) - - -class _HelpAction(Action): - - def __init__(self, - option_strings, - dest=SUPPRESS, - default=SUPPRESS, - help=None): - super(_HelpAction, self).__init__( - option_strings=option_strings, - dest=dest, - default=default, - nargs=0, - help=help) - - def __call__(self, parser, namespace, values, option_string=None): - parser.print_help() - parser.exit() - - -class _VersionAction(Action): - - def __init__(self, - option_strings, - dest=SUPPRESS, - default=SUPPRESS, - help=None): - super(_VersionAction, self).__init__( - option_strings=option_strings, - dest=dest, - default=default, - nargs=0, - help=help) - - def __call__(self, parser, namespace, values, option_string=None): - parser.print_version() - parser.exit() - - -class _SubParsersAction(Action): - - class _ChoicesPseudoAction(Action): - - def __init__(self, name, help): - sup = super(_SubParsersAction._ChoicesPseudoAction, self) - sup.__init__(option_strings=[], dest=name, help=help) - - def __init__(self, - option_strings, - prog, - parser_class, - dest=SUPPRESS, - help=None, - metavar=None): - - self._prog_prefix = prog - self._parser_class = parser_class - self._name_parser_map = {} - self._choices_actions = [] - - super(_SubParsersAction, self).__init__( - option_strings=option_strings, - dest=dest, - nargs=PARSER, - choices=self._name_parser_map, - help=help, - metavar=metavar) - - def add_parser(self, name, **kwargs): - # set prog from the existing prefix - if kwargs.get('prog') is None: - kwargs['prog'] = '%s %s' % (self._prog_prefix, name) - - # create a pseudo-action to hold the choice help - if 'help' in kwargs: - help = kwargs.pop('help') - choice_action = self._ChoicesPseudoAction(name, help) - self._choices_actions.append(choice_action) - - # create the parser and add it to the map - parser = self._parser_class(**kwargs) - self._name_parser_map[name] = parser - return parser - - def _get_subactions(self): - return self._choices_actions - - def __call__(self, parser, namespace, values, option_string=None): - parser_name = values[0] - arg_strings = values[1:] - - # set the parser name if requested - if self.dest is not SUPPRESS: - setattr(namespace, self.dest, parser_name) - - # select the parser - try: - parser = self._name_parser_map[parser_name] - except KeyError: - tup = parser_name, ', '.join(self._name_parser_map) - msg = _('unknown parser %r (choices: %s)' % tup) - raise ArgumentError(self, msg) - - # parse all the remaining options into the namespace - parser.parse_args(arg_strings, namespace) - - -# ============== -# Type classes -# ============== - -class FileType(object): - """Factory for creating file object types - - Instances of FileType are typically passed as type= arguments to the - ArgumentParser add_argument() method. - - Keyword Arguments: - - mode -- A string indicating how the file is to be opened. Accepts the - same values as the builtin open() function. - - bufsize -- The file's desired buffer size. Accepts the same values as - the builtin open() function. - """ - - def __init__(self, mode='r', bufsize=None): - self._mode = mode - self._bufsize = bufsize - - def __call__(self, string): - # the special argument "-" means sys.std{in,out} - if string == '-': - if 'r' in self._mode: - return _sys.stdin - elif 'w' in self._mode: - return _sys.stdout - else: - msg = _('argument "-" with mode %r' % self._mode) - raise ValueError(msg) - - # all other arguments are used as file names - if self._bufsize: - return open(string, self._mode, self._bufsize) - else: - return open(string, self._mode) - - def __repr__(self): - args = [self._mode, self._bufsize] - args_str = ', '.join([repr(arg) for arg in args if arg is not None]) - return '%s(%s)' % (type(self).__name__, args_str) - -# =========================== -# Optional and Positional Parsing -# =========================== - -class Namespace(_AttributeHolder): - """Simple object for storing attributes. - - Implements equality by attribute names and values, and provides a simple - string representation. - """ - - def __init__(self, **kwargs): - for name in kwargs: - setattr(self, name, kwargs[name]) - - def __eq__(self, other): - return vars(self) == vars(other) - - def __ne__(self, other): - return not (self == other) - - -class _ActionsContainer(object): - - def __init__(self, - description, - prefix_chars, - argument_default, - conflict_handler): - super(_ActionsContainer, self).__init__() - - self.description = description - self.argument_default = argument_default - self.prefix_chars = prefix_chars - self.conflict_handler = conflict_handler - - # set up registries - self._registries = {} - - # register actions - self.register('action', None, _StoreAction) - self.register('action', 'store', _StoreAction) - self.register('action', 'store_const', _StoreConstAction) - self.register('action', 'store_true', _StoreTrueAction) - self.register('action', 'store_false', _StoreFalseAction) - self.register('action', 'append', _AppendAction) - self.register('action', 'append_const', _AppendConstAction) - self.register('action', 'count', _CountAction) - self.register('action', 'help', _HelpAction) - self.register('action', 'version', _VersionAction) - self.register('action', 'parsers', _SubParsersAction) - - # raise an exception if the conflict handler is invalid - self._get_handler() - - # action storage - self._actions = [] - self._option_string_actions = {} - - # groups - self._action_groups = [] - self._mutually_exclusive_groups = [] - - # defaults storage - self._defaults = {} - - # determines whether an "option" looks like a negative number - self._negative_number_matcher = _re.compile(r'^-\d+|-\d*.\d+$') - - # whether or not there are any optionals that look like negative - # numbers -- uses a list so it can be shared and edited - self._has_negative_number_optionals = [] - - # ==================== - # Registration methods - # ==================== - def register(self, registry_name, value, object): - registry = self._registries.setdefault(registry_name, {}) - registry[value] = object - - def _registry_get(self, registry_name, value, default=None): - return self._registries[registry_name].get(value, default) - - # ================================== - # Namespace default settings methods - # ================================== - def set_defaults(self, **kwargs): - self._defaults.update(kwargs) - - # if these defaults match any existing arguments, replace - # the previous default on the object with the new one - for action in self._actions: - if action.dest in kwargs: - action.default = kwargs[action.dest] - - # ======================= - # Adding argument actions - # ======================= - def add_argument(self, *args, **kwargs): - """ - add_argument(dest, ..., name=value, ...) - add_argument(option_string, option_string, ..., name=value, ...) - """ - - # if no positional args are supplied or only one is supplied and - # it doesn't look like an option string, parse a positional - # argument - chars = self.prefix_chars - if not args or len(args) == 1 and args[0][0] not in chars: - kwargs = self._get_positional_kwargs(*args, **kwargs) - - # otherwise, we're adding an optional argument - else: - kwargs = self._get_optional_kwargs(*args, **kwargs) - - # if no default was supplied, use the parser-level default - if 'default' not in kwargs: - dest = kwargs['dest'] - if dest in self._defaults: - kwargs['default'] = self._defaults[dest] - elif self.argument_default is not None: - kwargs['default'] = self.argument_default - - # create the action object, and add it to the parser - action_class = self._pop_action_class(kwargs) - action = action_class(**kwargs) - return self._add_action(action) - - def add_argument_group(self, *args, **kwargs): - group = _ArgumentGroup(self, *args, **kwargs) - self._action_groups.append(group) - return group - - def add_mutually_exclusive_group(self, **kwargs): - group = _MutuallyExclusiveGroup(self, **kwargs) - self._mutually_exclusive_groups.append(group) - return group - - def _add_action(self, action): - # resolve any conflicts - self._check_conflict(action) - - # add to actions list - self._actions.append(action) - action.container = self - - # index the action by any option strings it has - for option_string in action.option_strings: - self._option_string_actions[option_string] = action - - # set the flag if any option strings look like negative numbers - for option_string in action.option_strings: - if self._negative_number_matcher.match(option_string): - if not self._has_negative_number_optionals: - self._has_negative_number_optionals.append(True) - - # return the created action - return action - - def _remove_action(self, action): - self._actions.remove(action) - - def _add_container_actions(self, container): - # collect groups by titles - title_group_map = {} - for group in self._action_groups: - if group.title in title_group_map: - msg = _('cannot merge actions - two groups are named %r') - raise ValueError(msg % (group.title)) - title_group_map[group.title] = group - - # map each action to its group - group_map = {} - for group in container._action_groups: - - # if a group with the title exists, use that, otherwise - # create a new group matching the container's group - if group.title not in title_group_map: - title_group_map[group.title] = self.add_argument_group( - title=group.title, - description=group.description, - conflict_handler=group.conflict_handler) - - # map the actions to their new group - for action in group._group_actions: - group_map[action] = title_group_map[group.title] - - # add container's mutually exclusive groups - # NOTE: if add_mutually_exclusive_group ever gains title= and - # description= then this code will need to be expanded as above - for group in container._mutually_exclusive_groups: - mutex_group = self.add_mutually_exclusive_group( - required=group.required) - - # map the actions to their new mutex group - for action in group._group_actions: - group_map[action] = mutex_group - - # add all actions to this container or their group - for action in container._actions: - group_map.get(action, self)._add_action(action) - - def _get_positional_kwargs(self, dest, **kwargs): - # make sure required is not specified - if 'required' in kwargs: - msg = _("'required' is an invalid argument for positionals") - raise TypeError(msg) - - # mark positional arguments as required if at least one is - # always required - if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]: - kwargs['required'] = True - if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs: - kwargs['required'] = True - - # return the keyword arguments with no option strings - return dict(kwargs, dest=dest, option_strings=[]) - - def _get_optional_kwargs(self, *args, **kwargs): - # determine short and long option strings - option_strings = [] - long_option_strings = [] - for option_string in args: - # error on one-or-fewer-character option strings - if len(option_string) < 2: - msg = _('invalid option string %r: ' - 'must be at least two characters long') - raise ValueError(msg % option_string) - - # error on strings that don't start with an appropriate prefix - if not option_string[0] in self.prefix_chars: - msg = _('invalid option string %r: ' - 'must start with a character %r') - tup = option_string, self.prefix_chars - raise ValueError(msg % tup) - - # error on strings that are all prefix characters - if not (_set(option_string) - _set(self.prefix_chars)): - msg = _('invalid option string %r: ' - 'must contain characters other than %r') - tup = option_string, self.prefix_chars - raise ValueError(msg % tup) - - # strings starting with two prefix characters are long options - option_strings.append(option_string) - if option_string[0] in self.prefix_chars: - if option_string[1] in self.prefix_chars: - long_option_strings.append(option_string) - - # infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x' - dest = kwargs.pop('dest', None) - if dest is None: - if long_option_strings: - dest_option_string = long_option_strings[0] - else: - dest_option_string = option_strings[0] - dest = dest_option_string.lstrip(self.prefix_chars) - dest = dest.replace('-', '_') - - # return the updated keyword arguments - return dict(kwargs, dest=dest, option_strings=option_strings) - - def _pop_action_class(self, kwargs, default=None): - action = kwargs.pop('action', default) - return self._registry_get('action', action, action) - - def _get_handler(self): - # determine function from conflict handler string - handler_func_name = '_handle_conflict_%s' % self.conflict_handler - try: - return getattr(self, handler_func_name) - except AttributeError: - msg = _('invalid conflict_resolution value: %r') - raise ValueError(msg % self.conflict_handler) - - def _check_conflict(self, action): - - # find all options that conflict with this option - confl_optionals = [] - for option_string in action.option_strings: - if option_string in self._option_string_actions: - confl_optional = self._option_string_actions[option_string] - confl_optionals.append((option_string, confl_optional)) - - # resolve any conflicts - if confl_optionals: - conflict_handler = self._get_handler() - conflict_handler(action, confl_optionals) - - def _handle_conflict_error(self, action, conflicting_actions): - message = _('conflicting option string(s): %s') - conflict_string = ', '.join([option_string - for option_string, action - in conflicting_actions]) - raise ArgumentError(action, message % conflict_string) - - def _handle_conflict_resolve(self, action, conflicting_actions): - - # remove all conflicting options - for option_string, action in conflicting_actions: - - # remove the conflicting option - action.option_strings.remove(option_string) - self._option_string_actions.pop(option_string, None) - - # if the option now has no option string, remove it from the - # container holding it - if not action.option_strings: - action.container._remove_action(action) - - -class _ArgumentGroup(_ActionsContainer): - - def __init__(self, container, title=None, description=None, **kwargs): - # add any missing keyword arguments by checking the container - update = kwargs.setdefault - update('conflict_handler', container.conflict_handler) - update('prefix_chars', container.prefix_chars) - update('argument_default', container.argument_default) - super_init = super(_ArgumentGroup, self).__init__ - super_init(description=description, **kwargs) - - # group attributes - self.title = title - self._group_actions = [] - - # share most attributes with the container - self._registries = container._registries - self._actions = container._actions - self._option_string_actions = container._option_string_actions - self._defaults = container._defaults - self._has_negative_number_optionals = \ - container._has_negative_number_optionals - - def _add_action(self, action): - action = super(_ArgumentGroup, self)._add_action(action) - self._group_actions.append(action) - return action - - def _remove_action(self, action): - super(_ArgumentGroup, self)._remove_action(action) - self._group_actions.remove(action) - - -class _MutuallyExclusiveGroup(_ArgumentGroup): - - def __init__(self, container, required=False): - super(_MutuallyExclusiveGroup, self).__init__(container) - self.required = required - self._container = container - - def _add_action(self, action): - if action.required: - msg = _('mutually exclusive arguments must be optional') - raise ValueError(msg) - action = self._container._add_action(action) - self._group_actions.append(action) - return action - - def _remove_action(self, action): - self._container._remove_action(action) - self._group_actions.remove(action) - - -class ArgumentParser(_AttributeHolder, _ActionsContainer): - """Object for parsing command line strings into Python objects. - - Keyword Arguments: - - prog -- The name of the program (default: sys.argv[0]) - - usage -- A usage message (default: auto-generated from arguments) - - description -- A description of what the program does - - epilog -- Text following the argument descriptions - - version -- Add a -v/--version option with the given version string - - parents -- Parsers whose arguments should be copied into this one - - formatter_class -- HelpFormatter class for printing help messages - - prefix_chars -- Characters that prefix optional arguments - - fromfile_prefix_chars -- Characters that prefix files containing - additional arguments - - argument_default -- The default value for all arguments - - conflict_handler -- String indicating how to handle conflicts - - add_help -- Add a -h/-help option - """ - - def __init__(self, - prog=None, - usage=None, - description=None, - epilog=None, - version=None, - parents=[], - formatter_class=HelpFormatter, - prefix_chars='-', - fromfile_prefix_chars=None, - argument_default=None, - conflict_handler='error', - add_help=True): - - superinit = super(ArgumentParser, self).__init__ - superinit(description=description, - prefix_chars=prefix_chars, - argument_default=argument_default, - conflict_handler=conflict_handler) - - # default setting for prog - if prog is None: - prog = _os.path.basename(_sys.argv[0]) - - self.prog = prog - self.usage = usage - self.epilog = epilog - self.version = version - self.formatter_class = formatter_class - self.fromfile_prefix_chars = fromfile_prefix_chars - self.add_help = add_help - - add_group = self.add_argument_group - self._positionals = add_group(_('positional arguments')) - self._optionals = add_group(_('optional arguments')) - self._subparsers = None - - # register types - def identity(string): - return string - self.register('type', None, identity) - - # add help and version arguments if necessary - # (using explicit default to override global argument_default) - if self.add_help: - self.add_argument( - '-h', '--help', action='help', default=SUPPRESS, - help=_('show this help message and exit')) - if self.version: - self.add_argument( - '-v', '--version', action='version', default=SUPPRESS, - help=_("show program's version number and exit")) - - # add parent arguments and defaults - for parent in parents: - self._add_container_actions(parent) - try: - defaults = parent._defaults - except AttributeError: - pass - else: - self._defaults.update(defaults) - - # ======================= - # Pretty __repr__ methods - # ======================= - def _get_kwargs(self): - names = [ - 'prog', - 'usage', - 'description', - 'version', - 'formatter_class', - 'conflict_handler', - 'add_help', - ] - return [(name, getattr(self, name)) for name in names] - - # ================================== - # Optional/Positional adding methods - # ================================== - def add_subparsers(self, **kwargs): - if self._subparsers is not None: - self.error(_('cannot have multiple subparser arguments')) - - # add the parser class to the arguments if it's not present - kwargs.setdefault('parser_class', type(self)) - - if 'title' in kwargs or 'description' in kwargs: - title = _(kwargs.pop('title', 'subcommands')) - description = _(kwargs.pop('description', None)) - self._subparsers = self.add_argument_group(title, description) - else: - self._subparsers = self._positionals - - # prog defaults to the usage message of this parser, skipping - # optional arguments and with no "usage:" prefix - if kwargs.get('prog') is None: - formatter = self._get_formatter() - positionals = self._get_positional_actions() - groups = self._mutually_exclusive_groups - formatter.add_usage(self.usage, positionals, groups, '') - kwargs['prog'] = formatter.format_help().strip() - - # create the parsers action and add it to the positionals list - parsers_class = self._pop_action_class(kwargs, 'parsers') - action = parsers_class(option_strings=[], **kwargs) - self._subparsers._add_action(action) - - # return the created parsers action - return action - - def _add_action(self, action): - if action.option_strings: - self._optionals._add_action(action) - else: - self._positionals._add_action(action) - return action - - def _get_optional_actions(self): - return [action - for action in self._actions - if action.option_strings] - - def _get_positional_actions(self): - return [action - for action in self._actions - if not action.option_strings] - - # ===================================== - # Command line argument parsing methods - # ===================================== - def parse_args(self, args=None, namespace=None): - args, argv = self.parse_known_args(args, namespace) - if argv: - msg = _('unrecognized arguments: %s') - self.error(msg % ' '.join(argv)) - return args - - def parse_known_args(self, args=None, namespace=None): - # args default to the system args - if args is None: - args = _sys.argv[1:] - - # default Namespace built from parser defaults - if namespace is None: - namespace = Namespace() - - # add any action defaults that aren't present - for action in self._actions: - if action.dest is not SUPPRESS: - if not hasattr(namespace, action.dest): - if action.default is not SUPPRESS: - default = action.default - if isinstance(action.default, _basestring): - default = self._get_value(action, default) - setattr(namespace, action.dest, default) - - # add any parser defaults that aren't present - for dest in self._defaults: - if not hasattr(namespace, dest): - setattr(namespace, dest, self._defaults[dest]) - - # parse the arguments and exit if there are any errors - try: - return self._parse_known_args(args, namespace) - except ArgumentError: - err = _sys.exc_info()[1] - self.error(str(err)) - - def _parse_known_args(self, arg_strings, namespace): - # replace arg strings that are file references - if self.fromfile_prefix_chars is not None: - arg_strings = self._read_args_from_files(arg_strings) - - # map all mutually exclusive arguments to the other arguments - # they can't occur with - action_conflicts = {} - for mutex_group in self._mutually_exclusive_groups: - group_actions = mutex_group._group_actions - for i, mutex_action in enumerate(mutex_group._group_actions): - conflicts = action_conflicts.setdefault(mutex_action, []) - conflicts.extend(group_actions[:i]) - conflicts.extend(group_actions[i + 1:]) - - # find all option indices, and determine the arg_string_pattern - # which has an 'O' if there is an option at an index, - # an 'A' if there is an argument, or a '-' if there is a '--' - option_string_indices = {} - arg_string_pattern_parts = [] - arg_strings_iter = iter(arg_strings) - for i, arg_string in enumerate(arg_strings_iter): - - # all args after -- are non-options - if arg_string == '--': - arg_string_pattern_parts.append('-') - for arg_string in arg_strings_iter: - arg_string_pattern_parts.append('A') - - # otherwise, add the arg to the arg strings - # and note the index if it was an option - else: - option_tuple = self._parse_optional(arg_string) - if option_tuple is None: - pattern = 'A' - else: - option_string_indices[i] = option_tuple - pattern = 'O' - arg_string_pattern_parts.append(pattern) - - # join the pieces together to form the pattern - arg_strings_pattern = ''.join(arg_string_pattern_parts) - - # converts arg strings to the appropriate and then takes the action - seen_actions = _set() - seen_non_default_actions = _set() - - def take_action(action, argument_strings, option_string=None): - seen_actions.add(action) - argument_values = self._get_values(action, argument_strings) - - # error if this argument is not allowed with other previously - # seen arguments, assuming that actions that use the default - # value don't really count as "present" - if argument_values is not action.default: - seen_non_default_actions.add(action) - for conflict_action in action_conflicts.get(action, []): - if conflict_action in seen_non_default_actions: - msg = _('not allowed with argument %s') - action_name = _get_action_name(conflict_action) - raise ArgumentError(action, msg % action_name) - - # take the action if we didn't receive a SUPPRESS value - # (e.g. from a default) - if argument_values is not SUPPRESS: - action(self, namespace, argument_values, option_string) - - # function to convert arg_strings into an optional action - def consume_optional(start_index): - - # get the optional identified at this index - option_tuple = option_string_indices[start_index] - action, option_string, explicit_arg = option_tuple - - # identify additional optionals in the same arg string - # (e.g. -xyz is the same as -x -y -z if no args are required) - match_argument = self._match_argument - action_tuples = [] - while True: - - # if we found no optional action, skip it - if action is None: - extras.append(arg_strings[start_index]) - return start_index + 1 - - # if there is an explicit argument, try to match the - # optional's string arguments to only this - if explicit_arg is not None: - arg_count = match_argument(action, 'A') - - # if the action is a single-dash option and takes no - # arguments, try to parse more single-dash options out - # of the tail of the option string - chars = self.prefix_chars - if arg_count == 0 and option_string[1] not in chars: - action_tuples.append((action, [], option_string)) - for char in self.prefix_chars: - option_string = char + explicit_arg[0] - explicit_arg = explicit_arg[1:] or None - optionals_map = self._option_string_actions - if option_string in optionals_map: - action = optionals_map[option_string] - break - else: - msg = _('ignored explicit argument %r') - raise ArgumentError(action, msg % explicit_arg) - - # if the action expect exactly one argument, we've - # successfully matched the option; exit the loop - elif arg_count == 1: - stop = start_index + 1 - args = [explicit_arg] - action_tuples.append((action, args, option_string)) - break - - # error if a double-dash option did not use the - # explicit argument - else: - msg = _('ignored explicit argument %r') - raise ArgumentError(action, msg % explicit_arg) - - # if there is no explicit argument, try to match the - # optional's string arguments with the following strings - # if successful, exit the loop - else: - start = start_index + 1 - selected_patterns = arg_strings_pattern[start:] - arg_count = match_argument(action, selected_patterns) - stop = start + arg_count - args = arg_strings[start:stop] - action_tuples.append((action, args, option_string)) - break - - # add the Optional to the list and return the index at which - # the Optional's string args stopped - assert action_tuples - for action, args, option_string in action_tuples: - take_action(action, args, option_string) - return stop - - # the list of Positionals left to be parsed; this is modified - # by consume_positionals() - positionals = self._get_positional_actions() - - # function to convert arg_strings into positional actions - def consume_positionals(start_index): - # match as many Positionals as possible - match_partial = self._match_arguments_partial - selected_pattern = arg_strings_pattern[start_index:] - arg_counts = match_partial(positionals, selected_pattern) - - # slice off the appropriate arg strings for each Positional - # and add the Positional and its args to the list - for action, arg_count in zip(positionals, arg_counts): - args = arg_strings[start_index: start_index + arg_count] - start_index += arg_count - take_action(action, args) - - # slice off the Positionals that we just parsed and return the - # index at which the Positionals' string args stopped - positionals[:] = positionals[len(arg_counts):] - return start_index - - # consume Positionals and Optionals alternately, until we have - # passed the last option string - extras = [] - start_index = 0 - if option_string_indices: - max_option_string_index = max(option_string_indices) - else: - max_option_string_index = -1 - while start_index <= max_option_string_index: - - # consume any Positionals preceding the next option - next_option_string_index = min([ - index - for index in option_string_indices - if index >= start_index]) - if start_index != next_option_string_index: - positionals_end_index = consume_positionals(start_index) - - # only try to parse the next optional if we didn't consume - # the option string during the positionals parsing - if positionals_end_index > start_index: - start_index = positionals_end_index - continue - else: - start_index = positionals_end_index - - # if we consumed all the positionals we could and we're not - # at the index of an option string, there were extra arguments - if start_index not in option_string_indices: - strings = arg_strings[start_index:next_option_string_index] - extras.extend(strings) - start_index = next_option_string_index - - # consume the next optional and any arguments for it - start_index = consume_optional(start_index) - - # consume any positionals following the last Optional - stop_index = consume_positionals(start_index) - - # if we didn't consume all the argument strings, there were extras - extras.extend(arg_strings[stop_index:]) - - # if we didn't use all the Positional objects, there were too few - # arg strings supplied. - if positionals: - self.error(_('too few arguments')) - - # make sure all required actions were present - for action in self._actions: - if action.required: - if action not in seen_actions: - name = _get_action_name(action) - self.error(_('argument %s is required') % name) - - # make sure all required groups had one option present - for group in self._mutually_exclusive_groups: - if group.required: - for action in group._group_actions: - if action in seen_non_default_actions: - break - - # if no actions were used, report the error - else: - names = [_get_action_name(action) - for action in group._group_actions - if action.help is not SUPPRESS] - msg = _('one of the arguments %s is required') - self.error(msg % ' '.join(names)) - - # return the updated namespace and the extra arguments - return namespace, extras - - def _read_args_from_files(self, arg_strings): - # expand arguments referencing files - new_arg_strings = [] - for arg_string in arg_strings: - - # for regular arguments, just add them back into the list - if arg_string[0] not in self.fromfile_prefix_chars: - new_arg_strings.append(arg_string) - - # replace arguments referencing files with the file content - else: - try: - args_file = open(arg_string[1:]) - try: - arg_strings = args_file.read().splitlines() - arg_strings = self._read_args_from_files(arg_strings) - new_arg_strings.extend(arg_strings) - finally: - args_file.close() - except IOError: - err = _sys.exc_info()[1] - self.error(str(err)) - - # return the modified argument list - return new_arg_strings - - def _match_argument(self, action, arg_strings_pattern): - # match the pattern for this action to the arg strings - nargs_pattern = self._get_nargs_pattern(action) - match = _re.match(nargs_pattern, arg_strings_pattern) - - # raise an exception if we weren't able to find a match - if match is None: - nargs_errors = { - None: _('expected one argument'), - OPTIONAL: _('expected at most one argument'), - ONE_OR_MORE: _('expected at least one argument'), - } - default = _('expected %s argument(s)') % action.nargs - msg = nargs_errors.get(action.nargs, default) - raise ArgumentError(action, msg) - - # return the number of arguments matched - return len(match.group(1)) - - def _match_arguments_partial(self, actions, arg_strings_pattern): - # progressively shorten the actions list by slicing off the - # final actions until we find a match - result = [] - for i in range(len(actions), 0, -1): - actions_slice = actions[:i] - pattern = ''.join([self._get_nargs_pattern(action) - for action in actions_slice]) - match = _re.match(pattern, arg_strings_pattern) - if match is not None: - result.extend([len(string) for string in match.groups()]) - break - - # return the list of arg string counts - return result - - def _parse_optional(self, arg_string): - # if it's an empty string, it was meant to be a positional - if not arg_string: - return None - - # if it doesn't start with a prefix, it was meant to be positional - if not arg_string[0] in self.prefix_chars: - return None - - # if it's just dashes, it was meant to be positional - if not arg_string.strip('-'): - return None - - # if the option string is present in the parser, return the action - if arg_string in self._option_string_actions: - action = self._option_string_actions[arg_string] - return action, arg_string, None - - # search through all possible prefixes of the option string - # and all actions in the parser for possible interpretations - option_tuples = self._get_option_tuples(arg_string) - - # if multiple actions match, the option string was ambiguous - if len(option_tuples) > 1: - options = ', '.join([option_string - for action, option_string, explicit_arg in option_tuples]) - tup = arg_string, options - self.error(_('ambiguous option: %s could match %s') % tup) - - # if exactly one action matched, this segmentation is good, - # so return the parsed action - elif len(option_tuples) == 1: - option_tuple, = option_tuples - return option_tuple - - # if it was not found as an option, but it looks like a negative - # number, it was meant to be positional - # unless there are negative-number-like options - if self._negative_number_matcher.match(arg_string): - if not self._has_negative_number_optionals: - return None - - # if it contains a space, it was meant to be a positional - if ' ' in arg_string: - return None - - # it was meant to be an optional but there is no such option - # in this parser (though it might be a valid option in a subparser) - return None, arg_string, None - - def _get_option_tuples(self, option_string): - result = [] - - # option strings starting with two prefix characters are only - # split at the '=' - chars = self.prefix_chars - if option_string[0] in chars and option_string[1] in chars: - if '=' in option_string: - option_prefix, explicit_arg = option_string.split('=', 1) - else: - option_prefix = option_string - explicit_arg = None - for option_string in self._option_string_actions: - if option_string.startswith(option_prefix): - action = self._option_string_actions[option_string] - tup = action, option_string, explicit_arg - result.append(tup) - - # single character options can be concatenated with their arguments - # but multiple character options always have to have their argument - # separate - elif option_string[0] in chars and option_string[1] not in chars: - option_prefix = option_string - explicit_arg = None - short_option_prefix = option_string[:2] - short_explicit_arg = option_string[2:] - - for option_string in self._option_string_actions: - if option_string == short_option_prefix: - action = self._option_string_actions[option_string] - tup = action, option_string, short_explicit_arg - result.append(tup) - elif option_string.startswith(option_prefix): - action = self._option_string_actions[option_string] - tup = action, option_string, explicit_arg - result.append(tup) - - # shouldn't ever get here - else: - self.error(_('unexpected option string: %s') % option_string) - - # return the collected option tuples - return result - - def _get_nargs_pattern(self, action): - # in all examples below, we have to allow for '--' args - # which are represented as '-' in the pattern - nargs = action.nargs - - # the default (None) is assumed to be a single argument - if nargs is None: - nargs_pattern = '(-*A-*)' - - # allow zero or one arguments - elif nargs == OPTIONAL: - nargs_pattern = '(-*A?-*)' - - # allow zero or more arguments - elif nargs == ZERO_OR_MORE: - nargs_pattern = '(-*[A-]*)' - - # allow one or more arguments - elif nargs == ONE_OR_MORE: - nargs_pattern = '(-*A[A-]*)' - - # allow one argument followed by any number of options or arguments - elif nargs is PARSER: - nargs_pattern = '(-*A[-AO]*)' - - # all others should be integers - else: - nargs_pattern = '(-*%s-*)' % '-*'.join('A' * nargs) - - # if this is an optional action, -- is not allowed - if action.option_strings: - nargs_pattern = nargs_pattern.replace('-*', '') - nargs_pattern = nargs_pattern.replace('-', '') - - # return the pattern - return nargs_pattern - - # ======================== - # Value conversion methods - # ======================== - def _get_values(self, action, arg_strings): - # for everything but PARSER args, strip out '--' - if action.nargs is not PARSER: - arg_strings = [s for s in arg_strings if s != '--'] - - # optional argument produces a default when not present - if not arg_strings and action.nargs == OPTIONAL: - if action.option_strings: - value = action.const - else: - value = action.default - if isinstance(value, _basestring): - value = self._get_value(action, value) - self._check_value(action, value) - - # when nargs='*' on a positional, if there were no command-line - # args, use the default if it is anything other than None - elif (not arg_strings and action.nargs == ZERO_OR_MORE and - not action.option_strings): - if action.default is not None: - value = action.default - else: - value = arg_strings - self._check_value(action, value) - - # single argument or optional argument produces a single value - elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]: - arg_string, = arg_strings - value = self._get_value(action, arg_string) - self._check_value(action, value) - - # PARSER arguments convert all values, but check only the first - elif action.nargs is PARSER: - value = [self._get_value(action, v) for v in arg_strings] - self._check_value(action, value[0]) - - # all other types of nargs produce a list - else: - value = [self._get_value(action, v) for v in arg_strings] - for v in value: - self._check_value(action, v) - - # return the converted value - return value - - def _get_value(self, action, arg_string): - type_func = self._registry_get('type', action.type, action.type) - if not hasattr(type_func, '__call__'): - if not hasattr(type_func, '__bases__'): # classic classes - msg = _('%r is not callable') - raise ArgumentError(action, msg % type_func) - - # convert the value to the appropriate type - try: - result = type_func(arg_string) - - # TypeErrors or ValueErrors indicate errors - except (TypeError, ValueError): - name = getattr(action.type, '__name__', repr(action.type)) - msg = _('invalid %s value: %r') - raise ArgumentError(action, msg % (name, arg_string)) - - # return the converted value - return result - - def _check_value(self, action, value): - # converted value must be one of the choices (if specified) - if action.choices is not None and value not in action.choices: - tup = value, ', '.join(map(repr, action.choices)) - msg = _('invalid choice: %r (choose from %s)') % tup - raise ArgumentError(action, msg) - - # ======================= - # Help-formatting methods - # ======================= - def format_usage(self): - formatter = self._get_formatter() - formatter.add_usage(self.usage, self._actions, - self._mutually_exclusive_groups) - return formatter.format_help() - - def format_help(self): - formatter = self._get_formatter() - - # usage - formatter.add_usage(self.usage, self._actions, - self._mutually_exclusive_groups) - - # description - formatter.add_text(self.description) - - # positionals, optionals and user-defined groups - for action_group in self._action_groups: - formatter.start_section(action_group.title) - formatter.add_text(action_group.description) - formatter.add_arguments(action_group._group_actions) - formatter.end_section() - - # epilog - formatter.add_text(self.epilog) - - # determine help from format above - return formatter.format_help() - - def format_version(self): - formatter = self._get_formatter() - formatter.add_text(self.version) - return formatter.format_help() - - def _get_formatter(self): - return self.formatter_class(prog=self.prog) - - # ===================== - # Help-printing methods - # ===================== - def print_usage(self, file=None): - self._print_message(self.format_usage(), file) - - def print_help(self, file=None): - self._print_message(self.format_help(), file) - - def print_version(self, file=None): - self._print_message(self.format_version(), file) - - def _print_message(self, message, file=None): - if message: - if file is None: - file = _sys.stderr - file.write(message) - - # =============== - # Exiting methods - # =============== - def exit(self, status=0, message=None): - if message: - _sys.stderr.write(message) - _sys.exit(status) - - def error(self, message): - """error(message: string) - - Prints a usage message incorporating the message to stderr and - exits. - - If you override this in a subclass, it should not return -- it - should either exit or raise an exception. - """ - self.print_usage(_sys.stderr) - self.exit(2, _('%s: error: %s\n') % (self.prog, message)) diff --git a/naoki/backend.py b/naoki/backend.py new file mode 100644 index 0000000..0d3582b --- /dev/null +++ b/naoki/backend.py @@ -0,0 +1,319 @@ +#!/usr/bin/python + +import os + +import chroot +import util + +from constants import * + +__cache = { + "package_names" : None, + "group_names" : None, +} + +def get_package_names(toolchain=False): + if not __cache["package_names"]: + names = [] + for repo in get_repositories(toolchain): + names.extend(repo.package_names) + + __cache["package_names"] = sorted(names) + + return __cache["package_names"] + +def get_group_names(): + if not __cache["group_names"]: + groups = [] + for package in get_package_names(): + package = PackageInfo(package) + if not package.group in groups: + groups.append(package.group) + + __cache["group_names"] = sorted(groups) + + return __cache["group_names"] + +def find_package_name(name, toolchain=False): + if name in get_package_names(toolchain): + return name + + for package in get_package_names(toolchain): + if os.path.basename(package) == name: + return package + +def depsolve(packages, recursive=False): + deps = [] + for package in packages: + if not package in deps: + deps.append(package) + + if not recursive or not deps: + return deps + + while True: + length = len(deps) + for dep in deps[:]: + deps.extend(dep.dependencies) + + new_deps = [] + for dep in deps: + if not dep in new_deps: + new_deps.append(dep) + + deps = new_deps + + if length == len(deps): + break + + deps.sort() + return deps + +def deptree(packages): + ret = [packages] + + while True: + next = [] + stage = ret[-1][:] + for package in stage[:]: + for dep in package.info.dependencies_all: + if dep in ret[-1]: + stage.remove(package) + next.append(package) + break + + ret[-1] = stage + if next: + ret.append(next) + continue + + break + + return ret + +def depsort(packages): + ret = [] + for l1 in deptree(packages): + ret.extend(l1) + return ret + +class PackageInfo(object): + __data = {} + + def __init__(self, name): + self._name = name + + def __repr__(self): + return "<PackageInfo %s>" % self.name + + def get_data(self): + if not self.__data.has_key(self.name): + self.__data[self.name] = self.fetch() + + return self.__data[self.name] + + def set_data(self, data): + self.__data[self.name] = data + + _data = property(get_data, set_data) + + def fetch(self): + env = os.environ.copy() + env.update(config.environment) + env["PKGROOT"] = PKGSDIR + output = util.do("make -f %s" % self.filename, shell=True, + cwd=os.path.join(PKGSDIR, self.name), returnOutput=1, env=env) + + ret = {} + for line in output.splitlines(): + a = line.split("=", 1) + if not len(a) == 2: continue + key, val = a + ret[key] = val.strip("\"") + + ret["FINGERPRINT"] = self.fingerprint + + return ret + + def fmtstr(self, s): + return s % self.all + + def getPackage(self, naoki): + return Package(self.name, naoki) + + @property + def all(self): + return { + "description" : self.description, + "filename" : self.filename, + "fingerprint" : self.fingerprint, + "group" : self.group, + "license" : self.license, + "maintainer" : self.maintainer, + "name" : self.name, + "objects" : self.objects, + "patches" : self.patches, + "release" : self.release, + "summary" : self.summary, + "version" : self.version, + } + + def _dependencies(self, s, recursive=False): + c = s + "_CACHE" + if not self._data.has_key(c): + deps = [] + for name in self._data.get(s).split(" "): + name = find_package_name(name) + if name: + deps.append(Dependency(name)) + + self._data.update({c : depsolve(deps, recursive)}) + + return self._data.get(c) + + @property + def dependencies(self): + return self._dependencies("PKG_DEPENDENCIES") + + @property + def dependencies_build(self): + return self._dependencies("PKG_BUILD_DEPENDENCIES") + + @property + def dependencies_all(self): + return depsolve(self.dependencies + self.dependencies_build, recursive=True) + + @property + def description(self): + return self._data.get("PKG_DESCRIPTION") + + @property + def filename(self): + return os.path.join(PKGSDIR, self.name, os.path.basename(self.name)) + ".nm" + + @property + def fingerprint(self): + return "%d" % os.stat(self.filename).st_mtime + + @property + def group(self): + return self._data.get("PKG_GROUP") + + @property + def id(self): + return "%s-%s-%s" % (self.name, self.version, self.release) + + @property + def license(self): + return self._data.get("PKG_LICENSE") + + @property + def maintainer(self): + return self._data.get("PKG_MAINTAINER") + + @property + def name(self): + return self._name + + @property + def objects(self): + return self._data.get("PKG_OBJECTS").split(" ") + + @property + def package_files(self): + return self._data.get("PKG_PACKAGES_FILES").split(" ") + + @property + def patches(self): + return self._data.get("PKG_PATCHES").split(" ") + + @property + def release(self): + return self._data.get("PKG_REL") + + @property + def summary(self): + return self._data.get("PKG_SUMMARY") + + @property + def version(self): + return self._data.get("PKG_VER") + + +class Dependency(PackageInfo): + def __repr__(self): + return "<Dependency %s>" % self.name + + +class Package(object): + def __init__(self, name, naoki): + self.info = PackageInfo(name) + self.naoki = naoki + + #self.log.debug("Initialized package object %s" % name) + + def build(self): + environment = chroot.Environment(self) + environment.build() + + def download(self): + return "TODO" + files = self.info.objects + #self.log.info("Downloading %s..." % files) + download(self.info.objects) + + def extract(self, dest): + files = [os.path.join(PACKAGESDIR, file) for file in self.info.package_files] + if not files: + return + + self.log.debug("Extracting %s..." % files) + util.do("%s --root=%s %s" % (os.path.join(TOOLSDIR, "decompressor"), + dest, " ".join(files)), shell=True) + + @property + def log(self): + return self.naoki.logging.getBuildLogger(self.info.id) + + +def get_repositories(toolchain=False): + if toolchain: + return Repository("toolchain") + + repos = [] + for repo in os.listdir(PKGSDIR): + if os.path.isdir(os.path.join(PKGSDIR, repo)): + repos.append(repo) + + repos.remove("toolchain") + + return [Repository(repo) for repo in repos] + +class Repository(object): + def __init__(self, name): + self.name = name + + def __repr__(self): + return "<Repository %s>" % self.name + + @property + def packages(self): + packages = [] + for package in os.listdir(self.path): + package = PackageInfo(os.path.join(self.name, package)) + packages.append(package) + + return packages + + @property + def package_names(self): + return [package.name for package in self.packages] + + @property + def path(self): + return os.path.join(PKGSDIR, self.name) + +if __name__ == "__main__": + pi = PackageInfo("core/grub") + + print pi.dependencies diff --git a/naoki/chroot.py b/naoki/chroot.py index 0feda24..a1cacb0 100644 --- a/naoki/chroot.py +++ b/naoki/chroot.py @@ -15,6 +15,7 @@ from logger import getLog class Environment(object): def __init__(self, package): self.package = package + self.naoki = self.package.naoki self.arch = arches.current self.config = config @@ -44,19 +45,8 @@ class Environment(object): ] self.buildroot = "buildroot.%d" % random.randint(0, 1024) - self.log = None - self.__initialized = False - def init(self): - if self.__initialized: - return - self._init() - self.__initialized = True - - def _init(self): - self._setupLogging() - - self.log.info("Setting up environment %s..." % self.chrootPath()) + self.log.debug("Setting up environment %s..." % self.chrootPath()) if os.path.exists(self.chrootPath()): self.clean() @@ -106,7 +96,7 @@ class Environment(object): util.rm(self.chrootPath()) def make(self, target): - file = "/usr/src%s" % self.package.filename[len(BASEDIR):] + file = "/usr/src%s" % self.package.info.filename[len(BASEDIR):] return self.doChroot("make -C %s -f %s %s" % \ (os.path.dirname(file), file, target), shell=True) @@ -152,17 +142,7 @@ class Environment(object): return ret def chrootPath(self, *args): - return os.path.join(BUILDDIR, "environments", self.package.id, *args) - - def _setupLogging(self): - logfile = os.path.join(LOGDIR, self.package.id, "build.log") - if not os.path.exists(os.path.dirname(logfile)): - util.mkdir(os.path.dirname(logfile)) - self.log = logging.getLogger(self.package.id) - fh = logging.FileHandler(logfile) - fh.setFormatter(logging.Formatter("[%(levelname)s] %(message)s")) - fh.setLevel(logging.NOTSET) - self.log.addHandler(fh) + return os.path.join(BUILDDIR, "environments", self.package.info.id, *args) def _setupDev(self): # files in /dev @@ -242,20 +222,14 @@ class Environment(object): util.do(cmd, raiseExc=0, shell=True) def extractAll(self): - packages = self.package.deps + self.package.build_deps - for pkg in config["mandatory_packages"]: - pkg = package.find(pkg) - if not pkg in packages: - packages.append(pkg) - - packages = package.depsolve(packages, recursive=True) + packages = [p.getPackage(self.naoki) \ + for p in self.package.info.dependencies_all] - for pkg in packages: - pkg.extract(self.chrootPath()) + for package in packages: + package.extract(self.chrootPath()) def build(self): self.package.download() - self.init() try: self.make("package") @@ -267,6 +241,10 @@ class Environment(object): if config["cleanup_on_success"]: self.clean() + @property + def log(self): + return self.package.log + class Toolchain(object): def __init__(self, arch): diff --git a/naoki/logger.py b/naoki/logger.py index a73f5da..4aaea8d 100644 --- a/naoki/logger.py +++ b/naoki/logger.py @@ -2,9 +2,77 @@ import curses import logging +import logging.config +import logging.handlers import sys import time +# fix for python 2.4 logging module bug: +logging.raiseExceptions = 0 + +from constants import * + +class Logging(object): + def __init__(self, naoki): + self.naoki = naoki + + self.setup() + + def setup(self): + self.naoki.log = self.log = logging.getLogger() + + log_ini = config["log_config_file"] + if os.path.exists(log_ini): + logging.config.fileConfig(log_ini) + + if sys.stderr.isatty(): + curses.setupterm() + self.log.handlers[0].setFormatter(_ColorLogFormatter()) + + # Set default configuration + self.quiet(config["quiet"]) + + self.log.handlers[0].setLevel(logging.DEBUG) + logging.getLogger("naoki").propagate = 1 + + if not os.path.isdir(LOGDIR): + os.makedirs(LOGDIR) + fh = logging.handlers.RotatingFileHandler(config["log_file"], + maxBytes=10*1024**2, backupCount=6) + fh.setFormatter(logging.Formatter("[%(levelname)s] %(message)s")) + fh.setLevel(logging.NOTSET) + self.log.addHandler(fh) + + def quiet(self, val): + if val: + self.log.debug("Enabled quiet logging mode") + self.log.handlers[0].setLevel(logging.WARNING) + else: + #self.log.debug("Enabled verbose logging mode") + self.log.handlers[0].setLevel(logging.INFO) + + def _setupBuildLogger(self, logger): + logger.setLevel(logging.DEBUG) + logger.parent = self.log + logger.propagate = 1 + + handler = logging.handlers.RotatingFileHandler( + os.path.join(LOGDIR, logger.name + ".log"), maxBytes=10*1024**2, + backupCount=5) + + formatter = logging.Formatter("[BUILD] %(message)s") + handler.setFormatter(formatter) + + logger.addHandler(handler) + + def getBuildLogger(self, name): + logger = logging.getLogger(name) + if not logger.handlers: + self._setupBuildLogger(logger) + + return logger + + # defaults to module verbose log # does a late binding on log. Forwards all attributes to logger. # works around problem where reconfiguring the logging module means loggers @@ -42,8 +110,7 @@ class _ColorLogFormatter(logging.Formatter): record.message = "Bad message (%r): %r" % (e, record.__dict__) record.asctime = time.strftime( "%H:%M:%S", self.converter(record.created)) - prefix = '[%(levelname)7s | %(asctime)s]' % \ - record.__dict__ + prefix = " %(levelname)-7s" % record.__dict__ color = self._colors.get(record.levelno, self._normal) formatted = color + prefix + self._normal + " " + record.message if record.exc_info: diff --git a/naoki/terminal.py b/naoki/terminal.py index 49a5985..d39ee85 100644 --- a/naoki/terminal.py +++ b/naoki/terminal.py @@ -3,8 +3,260 @@ import fcntl import os import struct +import sys import termios +from constants import * + +class ParsingError(Exception): + pass + + +class NameSpace(dict): + def __getattr__(self, attr): + try: + return self.__getitem__(attr) + except KeyError: + raise AttributeError + + +class Parser(object): + def __init__(self, name, arguments=[], parsers=[]): + self.name = name + self.arguments = arguments + self.parsers = parsers + + self.subparser = None + + def __repr__(self): + return "<Parser %s>" % self.name + + def check(self, args): + if not args: + return False + + return self.name == args[0] + + def parse(self, args): + if args and self.name == args[0]: + args = args[1:] + + for argument in self.arguments: + args = argument.parse(args) + + if args and self.parsers: + for parser in self.parsers: + if not parser.check(args): + continue + + self.subparser = parser + break + + if self.subparser: + args = self.subparser.parse(args) + + return args + + @property + def values(self): + ret = NameSpace( + name=self.name, + ) + if self.subparser: + ret["action"] = self.subparser.values + + for argument in self.arguments: + ret[argument.name] = argument.value() + + return ret + + +class _Argument(object): + DEFAULT_HELP = "No help available" + + def __init__(self, name, args, **kwargs): + self.name = name + self.args = args + self.help = kwargs.get("help", self.DEFAULT_HELP) + + self._parsed = False + self._parsed_args = [] + + def parse(self, args): + raise NotImplementedError + + def value(self): + raise NotImplementedError + + +class Option(_Argument): + def parse(self, args): + self._parsed = True + + new_args = [] + for arg in args: + if arg in self.args: + self._parsed_args.append(arg) + else: + new_args.append(arg) + + return new_args + + def value(self): + for arg in self._parsed_args: + if arg in self.args: + return True + + return False + + +class Choice(_Argument): + def __init__(self, *args, **kwargs): + _Argument.__init__(self, *args, **kwargs) + + self.choices = kwargs.get("choices", []) + + def parse(self, args): + self._parsed = True + + new_args = [] + next_arg = False + for arg in args: + if next_arg: + if self.choices and not arg in self.choices: + raise ParsingError, "'%s' is not an allowed choice" % arg + + self._parsed_args.append(arg) + next_arg = False + continue + + if arg in self.args: + self._parsed_args.append(arg) + next_arg = True + else: + new_args.append(arg) + + return new_args + + def value(self): + if self._parsed_args: + return self._parsed_args[-1] + + return None + + +class List(_Argument): + def __init__(self, name, **kwargs): + _Argument.__init__(self, name, [], **kwargs) + + def parse(self, args): + self._parsed = True + self._parsed_args = args + + return [] + + def value(self): + return self._parsed_args + + +class Commandline(object): + def __init__(self, naoki): + self.naoki = naoki + + # Parse the stuff + self.args = self.__parse() + + # ... afterwards, process global directives + self.__process_global(self.args) + + def __process_global(self, args): + # Set quiet mode + self.naoki.logging.quiet(args.quiet) + + # XXX Set architecture + + def __parse(self): + parser = Parser("root", + arguments=[ + Option("help", ["-h", "--help"], help="Show help text"), + Option("quiet", ["-q", "--quiet"], help="Set quiet mode"), + Option("debug", ["-d", "--debug"], help="Set debugging mode"), + Choice("arch", ["-a", "--arch"], help="Set architecture", + choices=[arch for arch in arches.all]), + ], + parsers=[ + # Build + Parser("build", + arguments=[ + List("packages"), + ]), + + # Toolchain + Parser("toolchain", + parsers=[ + Parser("download"), + Parser("build"), + ]), + + # Package + Parser("package", + parsers=[ + Parser("info", + arguments=[ + Option("long", ["-l", "--long"]), + Option("machine", ["--machine"]), + Option("wiki", ["--wiki"]), + List("packages"), + ]), + Parser("tree"), + Parser("list", + arguments=[ + Option("long", ["-l", "--long"]), + ]), + Parser("groups", + arguments=[ + Option("wiki", ["--wiki"]), + ]), + ]), + + # Source + Parser("source", + parsers=[ + Parser("download", + arguments=[ + List("packages"), + ]), + Parser("upload", + arguments=[ + List("packages"), + ]), + Parser("clean"), + ]), + + # Check + Parser("check", + parsers=[ + Parser("host"), + ]), + + # Batch + Parser("batch", + parsers=[ + Parser("cron"), + ]), + ]) + + args = parser.parse(sys.argv[1:]) + + if args: + raise ParsingError, "Unknown argument(s) passed: %s" % args + + return parser.values + + def help(self): + print "PRINTING HELP TEXT" + + DEFAULT_COLUMNS = 80 DEFAULT_LINES = 25 @@ -44,3 +296,7 @@ def get_lines(): def get_columns(): return get_size()[1] + + +if __name__ == "__main__": + cl = Commandline(naoki=None) diff --git a/naoki/util.py b/naoki/util.py index d0dcf55..a514a8a 100644 --- a/naoki/util.py +++ b/naoki/util.py @@ -98,7 +98,7 @@ def logOutput(fds, logger, returnOutput=1, start=0, timeout=0): tail = lines.pop() for line in lines: if line == '': continue - logger.debug(line) + logger.info(line) for h in logger.handlers: h.flush() if returnOutput: hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. a00d6e8e1ba50103623c660a174e9878c92f1542
by git＠ipfire.org 18 Mar '10

18 Mar '10

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 3.x development tree branch, master, updated. 2832b22bf3dea1805f8ea87c46ec6f8bafce88db
by git＠ipfire.org 18 Mar '10

18 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 3.x development tree". The branch, master has been updated via 2832b22bf3dea1805f8ea87c46ec6f8bafce88db (commit) from 8742cef60d3378ab5f769f4fe7f7968b2240c334 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 2832b22bf3dea1805f8ea87c46ec6f8bafce88db Author: Schantl Stefan <Stevee(a)ipfire.org> Date: Sat Mar 13 17:11:52 2010 +0100 squasfs-tools: Add zlib as dependency. ----------------------------------------------------------------------- Summary of changes: pkgs/core/squashfs-tools/squashfs-tools.nm | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) Difference in files: diff --git a/pkgs/core/squashfs-tools/squashfs-tools.nm b/pkgs/core/squashfs-tools/squashfs-tools.nm index e440b2c..ae4f7fc 100644 --- a/pkgs/core/squashfs-tools/squashfs-tools.nm +++ b/pkgs/core/squashfs-tools/squashfs-tools.nm @@ -34,16 +34,14 @@ PKG_URL = http://squashfs.sf.net/ PKG_LICENSE = GPLv2+ PKG_SUMMARY = Utility for the creation of squashfs filesystems. +PKG_BUILD_DEPS+= zlib + define PKG_DESCRIPTION Squashfs is a highly compressed read-only filesystem for Linux. endef PKG_TARBALL = squashfs$(PKG_VER).tar.gz -############################################################################### -# Installation Details -############################################################################### - define STAGE_BUILD mv /usr/src/squashfs4.0 $(DIR_APP) cd $(DIR_APP)/$(PKG_NAME) && sed -e "s/^CFLAGS := /& $(CFLAGS) /" -i Makefile hooks/post-receive -- IPFire 3.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 2.x development tree branch, next, updated. f81179c3b21950bba2434b05a1f8b80b5567788c
by git＠ipfire.org 18 Mar '10

18 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 2.x development tree". The branch, next has been updated via f81179c3b21950bba2434b05a1f8b80b5567788c (commit) from 27b6a4693fe2cad3fe72c15a67fc96fb734190f0 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f81179c3b21950bba2434b05a1f8b80b5567788c Author: Arne Fitzenreiter <arne_f(a)ipfire.org> Date: Thu Mar 18 07:59:24 2010 +0100 Fix build of syslogdctrl. ----------------------------------------------------------------------- Summary of changes: src/misc-progs/syslogdctrl.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) Difference in files: diff --git a/src/misc-progs/syslogdctrl.c b/src/misc-progs/syslogdctrl.c index 0e10e16..a470e47 100644 --- a/src/misc-progs/syslogdctrl.c +++ b/src/misc-progs/syslogdctrl.c @@ -30,7 +30,8 @@ int main(void) { - char buffer[STRING_SIZE], command[STRING_SIZE], hostname[STRING_SIZE], varmessages[STRING_SIZE]; + char buffer[STRING_SIZE], command[STRING_SIZE], hostname[STRING_SIZE]; + char varmessages[STRING_SIZE], enable_asynclog[STRING_SIZE]; int config_fd,rc,fd,pid; struct stat st; struct keyvalue *kv = NULL; @@ -62,6 +63,13 @@ int main(void) fprintf(stderr, "Cannot read REMOTELOG_ADDR\n"); exit(ERR_SETTINGS); } + + if (!findkey(kv, "ENABLE_ASYNCLOG", enable_asynclog)) + { + fprintf(stderr, "Cannot read ENABLE_ASYNCLOG\n"); + exit(ERR_SETTINGS); + } + if (!findkey(kv, "VARMESSAGES", varmessages)) { @@ -125,7 +133,7 @@ int main(void) /* Replace the logging option*/ safe_system("grep -v '/var/log/messages' < /etc/syslog.conf.new > /etc/syslog.conf.tmp && mv /etc/syslog.conf.tmp /etc/syslog.conf.new"); - if (strcmp(ENABLE_ASYNCLOG,"on")) + if (strcmp(enable_asynclog,"on")) snprintf(command, STRING_SIZE-1, "printf '%s -/var/log/messages' >> /etc/syslog.conf.new", varmessages ); else snprintf(command, STRING_SIZE-1, "printf '%s /var/log/messages' >> /etc/syslog.conf.new", varmessages ); hooks/post-receive -- IPFire 2.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 2.x development tree branch, next, updated. 27b6a4693fe2cad3fe72c15a67fc96fb734190f0
by git＠ipfire.org 16 Mar '10

16 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 2.x development tree". The branch, next has been updated via 27b6a4693fe2cad3fe72c15a67fc96fb734190f0 (commit) from ddd1589dabae74ed08ff00c51355ef7f3243ccc2 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 27b6a4693fe2cad3fe72c15a67fc96fb734190f0 Author: Christian Schmidt <maniacikarus(a)ipfire.org> Date: Tue Mar 16 21:23:09 2010 +0100 Fixed the log settings and added translation. ----------------------------------------------------------------------- Summary of changes: html/cgi-bin/logs.cgi/config.dat | 4 +++- langs/de/cgi-bin/de.pl | 1 + langs/en/cgi-bin/en.pl | 1 + langs/fr/cgi-bin/fr.pl | 1 + 4 files changed, 6 insertions(+), 1 deletions(-) Difference in files: diff --git a/html/cgi-bin/logs.cgi/config.dat b/html/cgi-bin/logs.cgi/config.dat index 392edac..1f97a17 100644 --- a/html/cgi-bin/logs.cgi/config.dat +++ b/html/cgi-bin/logs.cgi/config.dat @@ -155,7 +155,9 @@ END print <<END <table width='100%'> <tr> - <td class='base'>$Lang::tr{'enabled'}</td><td><input type='checkbox' name='ENABLE_ASYNCLOG' $checked{'ENABLE_ASYNCLOG'}{'on'} /></td> + <td class='base' colspan='2'><input type='checkbox' name='ENABLE_ASYNCLOG' $checked{'ENABLE_ASYNCLOG'}{'on'} />$Lang::tr{'Async logging enabled'}</td> +<tr> +</tr> <td>$Lang::tr{'log var messages'}</td><td><input type='text' name='VARMESSAGES' size='50' value='$logsettings{'VARMESSAGES'}' /></td> </tr> </table> diff --git a/langs/de/cgi-bin/de.pl b/langs/de/cgi-bin/de.pl index e18fdfb..6772aec 100644 --- a/langs/de/cgi-bin/de.pl +++ b/langs/de/cgi-bin/de.pl @@ -4,6 +4,7 @@ 'Act as' => 'Konfiguriert als', 'Add Port Rule' => 'Port Regel hinzufÃŒgen', 'Add Rule' => 'Regel hinzufÃŒgen', +'Async logging enabled' => 'Aktiviere asynchrones schreiben des Syslogs', 'optional at cmd' => 'zusÃ€tzlicher Modembefehl', 'Choose Rule' => 'WÃ€hlen sie <u>eine</u> der untenstehenden Regeln aus.', 'Class' => 'Klasse', diff --git a/langs/en/cgi-bin/en.pl b/langs/en/cgi-bin/en.pl index 0bc5033..9a4d590 100644 --- a/langs/en/cgi-bin/en.pl +++ b/langs/en/cgi-bin/en.pl @@ -4,6 +4,7 @@ 'Act as' => 'Act as:', 'Add Port Rule' => 'Add port rule', 'Add Rule' => 'Add rule', +'Async logging enabled' => 'Enable asynchronous writing of the syslog file', 'optional at cmd' => 'optional AT command', 'Choose Rule' => 'Choose <u>one</u> of the following rules.', 'Class' => 'Class', diff --git a/langs/fr/cgi-bin/fr.pl b/langs/fr/cgi-bin/fr.pl index 0bc5033..a208d61 100644 --- a/langs/fr/cgi-bin/fr.pl +++ b/langs/fr/cgi-bin/fr.pl @@ -4,6 +4,7 @@ 'Act as' => 'Act as:', 'Add Port Rule' => 'Add port rule', 'Add Rule' => 'Add rule', +'Async logging enabled' => 'Activer écriture asynchrone de syslogs', 'optional at cmd' => 'optional AT command', 'Choose Rule' => 'Choose <u>one</u> of the following rules.', 'Class' => 'Class', hooks/post-receive -- IPFire 2.x development tree

1 0

[IPFire-SCM] [git.ipfire.org] IPFire 2.x development tree branch, next, updated. ddd1589dabae74ed08ff00c51355ef7f3243ccc2
by git＠ipfire.org 16 Mar '10

16 Mar '10

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 2.x development tree". The branch, next has been updated via ddd1589dabae74ed08ff00c51355ef7f3243ccc2 (commit) via 6945083ea536800184d56cab513176b5718b9eaf (commit) from 0cc5df4e29dc872bdcb475875077f4339dc0d435 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ddd1589dabae74ed08ff00c51355ef7f3243ccc2 Author: Christian Schmidt <maniacikarus(a)ipfire.org> Date: Tue Mar 16 21:11:13 2010 +0100 Changed installer. We now support ext2 installation and dropped reiser4. System now calculates a smaller swap partition since there is no more need to have this large swapfiles with a large amount of total ram available. commit 6945083ea536800184d56cab513176b5718b9eaf Author: Christian Schmidt <maniacikarus(a)ipfire.org> Date: Tue Mar 16 21:09:44 2010 +0100 Added ability to change syslog to async logging. ----------------------------------------------------------------------- Summary of changes: html/cgi-bin/logs.cgi/config.dat | 6 ++++++ src/install+setup/install/main.c | 26 +++++++++++++------------- src/misc-progs/syslogdctrl.c | 6 +++++- 3 files changed, 24 insertions(+), 14 deletions(-) Difference in files: diff --git a/html/cgi-bin/logs.cgi/config.dat b/html/cgi-bin/logs.cgi/config.dat index f39981e..392edac 100644 --- a/html/cgi-bin/logs.cgi/config.dat +++ b/html/cgi-bin/logs.cgi/config.dat @@ -32,6 +32,7 @@ $logsettings{'LOGWATCH_LEVEL'} = 'Low'; $logsettings{'LOGWATCH_KEEP'} = '56'; my @VS = ('15','50','100','150','250','500'); $logsettings{'ENABLE_REMOTELOG'} = 'off'; +$logsettings{'ENABLE_ASYNCLOG'} = 'off'; $logsettings{'REMOTELOG_ADDR'} = ''; $logsettings{'VARMESSAGES'} = 'cron.none;daemon.*;local0.*;local2.*;*.info;mail.none;authpriv.*'; $logsettings{'ACTION'} = ''; @@ -66,6 +67,10 @@ if ($logsettings{'ACTION'} eq $Lang::tr{'save'}) &General::readhash("${General::swroot}/logging/settings", \%logsettings); +$checked{'ENABLE_ASYNCLOG'}{'off'} = ''; +$checked{'ENABLE_ASYNCLOG'}{'on'} = ''; +$checked{'ENABLE_ASYNCLOG'}{$logsettings{'ENABLE_ASYNCLOG'}} = "checked='checked'"; + $checked{'ENABLE_REMOTELOG'}{'off'} = ''; $checked{'ENABLE_REMOTELOG'}{'on'} = ''; $checked{'ENABLE_REMOTELOG'}{$logsettings{'ENABLE_REMOTELOG'}} = "checked='checked'"; @@ -150,6 +155,7 @@ END print <<END <table width='100%'> <tr> + <td class='base'>$Lang::tr{'enabled'}</td><td><input type='checkbox' name='ENABLE_ASYNCLOG' $checked{'ENABLE_ASYNCLOG'}{'on'} /></td> <td>$Lang::tr{'log var messages'}</td><td><input type='text' name='VARMESSAGES' size='50' value='$logsettings{'VARMESSAGES'}' /></td> </tr> </table> diff --git a/src/install+setup/install/main.c b/src/install+setup/install/main.c index a1dd90d..c16bde8 100644 --- a/src/install+setup/install/main.c +++ b/src/install+setup/install/main.c @@ -14,9 +14,9 @@ #define INST_FILECOUNT 6200 #define UNATTENDED_CONF "/cdrom/boot/unattended.conf" -#define REISER4 0 -#define REISERFS 1 -#define EXT3 2 +#define EXT2 0 +#define REISERFS 2 +#define EXT3 1 FILE *flog = NULL; char *mylog; @@ -43,7 +43,7 @@ int main(int argc, char *argv[]) int rc = 0; char commandstring[STRING_SIZE]; char mkfscommand[STRING_SIZE]; - char *fstypes[] = { "Reiser4", "ReiserFS", "ext3", NULL }; + char *fstypes[] = { "ext2", "ext3", "ReiserFS", NULL }; int fstype = REISERFS; int choice; int i; @@ -304,10 +304,10 @@ int main(int argc, char *argv[]) /* Calculating Swap-Size dependend of Ram Size */ if (memory < 128) swap_file = 32; - else if (memory >= 1024) - swap_file = 512; + else if (memory > 512) + swap_file = 256; else - swap_file = memory; + swap_file = memory / 4; /* Calculating Root-Size dependend of Max Disk Space */ if ( disk < 756 ) @@ -366,9 +366,9 @@ int main(int argc, char *argv[]) goto EXIT; } - if (fstype == REISER4) { - mysystem("/sbin/modprobe reiser4"); - sprintf(mkfscommand, "/sbin/mkfs.reiser4 -y"); + if (fstype == EXT2) { + mysystem("/sbin/modprobe ext2"); + sprintf(mkfscommand, "/sbin/mke2fs -T ext2 -c"); } else if (fstype == REISERFS) { mysystem("/sbin/modprobe reiserfs"); sprintf(mkfscommand, "/sbin/mkreiserfs -f"); @@ -483,9 +483,9 @@ int main(int argc, char *argv[]) /* Update /etc/fstab */ replace("/harddisk/etc/fstab", "DEVICE", hdparams.devnode_part_run); - if (fstype == REISER4) { - replace("/harddisk/etc/fstab", "FSTYPE", "reiser4"); - replace("/harddisk/boot/grub/grub.conf", "MOUNT", "rw"); + if (fstype == EXT2) { + replace("/harddisk/etc/fstab", "FSTYPE", "ext2"); + replace("/harddisk/boot/grub/grub.conf", "MOUNT", "ro"); } else if (fstype == REISERFS) { replace("/harddisk/etc/fstab", "FSTYPE", "reiserfs"); replace("/harddisk/boot/grub/grub.conf", "MOUNT", "ro"); diff --git a/src/misc-progs/syslogdctrl.c b/src/misc-progs/syslogdctrl.c index 9880ee2..0e10e16 100644 --- a/src/misc-progs/syslogdctrl.c +++ b/src/misc-progs/syslogdctrl.c @@ -123,9 +123,13 @@ int main(void) close(config_fd); /* Replace the logging option*/ - safe_system("grep -v '/var/log/messages' < /etc/syslog.conf.new > /etc/syslog.conf.tmp && mv /etc/syslog.conf.tmp /etc/syslog.conf.new"); + + if (strcmp(ENABLE_ASYNCLOG,"on")) + snprintf(command, STRING_SIZE-1, "printf '%s -/var/log/messages' >> /etc/syslog.conf.new", varmessages ); + else snprintf(command, STRING_SIZE-1, "printf '%s /var/log/messages' >> /etc/syslog.conf.new", varmessages ); + safe_system(command); if (rename("/etc/syslog.conf.new", "/etc/syslog.conf") == -1) hooks/post-receive -- IPFire 2.x development tree

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

IPFire-SCM