### TODO On reboot, need to: /etc/init.d/network restart mount /cgroup/ lxc-start -n wheezy -f /data/wheezy/config -d lxc-start -n social -f /data/social/config -d lxc-start -n library -f /data/library/config -d ### Recompiling Generic OpenWRT Attitude Adjustment from upstream Note: can skip the patching below if you just use http://github.com/bnewbold/openwrt branch "soekris" and copy rooter.config to .config. Note: GRUB2-based builds from trunk don't seem to work, so make sure the Attitude Adjustment branch is used and the grub2 build option is not enabled. The official OpenWRT build documentation contains a set of host-system package dependancies (build-essential, etc): http://wiki.openwrt.org/doc/howto/buildroot.exigence http://wiki.openwrt.org/doc/howto/build Git clone: git://nbd.name/openwrt.git In base directory, fetch package feeds: ./scripts/feeds update -a # TODO: maybe this is too much? ./scripts/feeds install -a # or, ./scripts/feeds install -d m build-essential Apply the two following patches (unless they have been unstreamed) (patch -p0 < file.patch): https://dev.openwrt.org/ticket/10525 https://dev.openwrt.org/ticket/12262 "make menuconfig", select x86 and net6501 as target, save config, run "make defconfig" to automatically setup target specific stuff (?). "make kernel_menuconfig CONFIG_TARGET=subtarget" and ensure the following is checked: Device Drivers ---> <*> Serial ATA and Parallel ATA drivers ---> [*] AHCI SATA support ext2 no longer necessary by default... "Use ext4 for ext2/ext3 file systems"? ## rooter-Specific OpenWRT Build Tweaks In menuconfig: global build settings disable binary stripping advanced toolchain options eglibc (not ulibc) gcc 4.7.0 (not gcc 4.6.x with Linaro enhancements) target build settings: root fs archives: tar.gz root filesystem images: ext4 38400 serial port baud rate don't gzip images 16 kernel partition size (MB) 196 root partition size base system bridge libpthread qos-scripts wireless-tools ipv6 firewall traceroute6 ndisc6 6rd 6scripts ahcpd ipv6calc-mini radvd radvdump luci luci, luci-ssl luci-app-ahcp luci-app-ddns luci-app-diag-devinfo luci-app-qos luci-app-radvd luci-app-statistics kernel kmod-usb-acm kmod-usb-net kmod-usb-storage all intel wireless, as optional modules fs-vfat, fs-msdosfs ralink usb network version control git subversion client file transfer wget bmon ifconfig hostname mtr netstat nisdomainname netcat netperf netstat-nat ngrep utilities vim gzip gnupg e2fsprogs mkdosfs resize2fs tune2fs fdisk cfdisk lsblk hdparam bonniexx development build-essentials libraries libgmp (else gcc compile errors) TODO: actually get build-essentials compiling TODO: mkdosfs download link is dead if compiling build-essentials, might need: bnewbold@ziggy:~/code/openwrt_trunk$ cat /home/bnewbold/code/openwrt_trunk/toolchain/gcc/patches/4.7.0/209-automake-bullshit.patch --- a/config/override.m4 +++ b/config/override.m4 @@ -29,7 +29,7 @@ dnl Ensure exactly this Autoconf version is used m4_ifndef([_GCC_AUTOCONF_VERSION], - [m4_define([_GCC_AUTOCONF_VERSION], [2.64])]) + [m4_define([_GCC_AUTOCONF_VERSION], [2.68])]) dnl Test for the exact version when AC_INIT is expanded. dnl This allows to update the tree in steps (for testing) ## Install an OpenWRT image (first time) Use unetbootin to write a generic linux distro (like debian stable or arch linux) to a USB stick. Edit the syslinux.cfg file so it looks like: default menu.c32 default arch menu title UNetbootin timeout 30 serial 0 38400 console 0 prompt 0 label arch menu label Arch kernel /ubnkern append vga=normal initrd=/ubninit ../../ -- quiet console=ttyS0,38400 earlyprint=serial,ttyS0,38400 Copy over all the .img files required to the root directory of this device, then boot up (connect via serial to select boot device). Wait until you get to either a login or a prompt, then copy over the entire image to the device: # dd if=openwrt-x86-net6501-combined-squashfs.img of=/dev/sda bs=1M Reboot with all defaults! ## Misc OpenWRT notes To allow remote SSH logins on port 22, add the following custom firewall rule: iptables -t nat -A prerouting_wan -p tcp --dport 22 -j ACCEPT iptables -A input_wan -p tcp --dport 22 -j ACCEPT To allow all incoming requests to the router, add the following to /etc/config/firewall: config redirect option src wan option proto all option dest_ip 192.168.1.1 To do a sysupgrade, use a full combined .img file, eg scp and run: sysupgrade openwrt-x86-net6501-combined-squashfs.img NOTE: sysupgrade does not seem to work across ulibc/eglibc change? ## LXC Compilation notes Make sure the following packages are installed (if they weren't from above): lsblk perl file vim util > coreutils admin > debootstrap (has requirements) block-mount Select the "getopt" function in busybox: base system > busybox > getopt and support option -l Put the lxc and attr packages in ./package, then run: ./scripts/feeds install libattr ./scripts/feeds install libcap ./scripts/feeds install lxc Do menuconfig and select these (in Utilities), and also the bash shell. To compile individual packages, try: make package/lxc/compile make package/lxc/install make package/index NOTE: "cgroup namespace support" is not an option in kernel configuration any more, but shows up as a requirement in lxc-checkconfig. LXC works regardless, may be some security issues though? See also: https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/827798 ## LXC Configuration notes Ok, once all the above is compiled and loaded up, ready to configure the OpenWRT host. Create /cgroups directory and a mount point like /data. Edit /etc/conf/fstab: config 'mount' option 'target' '/data' option 'device' '/dev/sda3' option 'fstype' 'ext4' option 'options' 'rw,sync' option 'enabled' '1' option 'enabled_fsck' '1' config 'mount' option 'target' '/cgroup' option 'device' 'cgroup' option 'fstype' 'cgroup' option 'options' 'none' option 'enabled' '1' option 'enabled_fsck' '0' Enable block-mount ("/etc/init.d/fstab enable") and start it ("/etc/init.d/fstab start"). Copy the devices.tar.gz file to /usr/share/debootstrap/devices.tar.gz. Next build a debian rootfs. This is rather slow the first time around. Make sure /data is mounted, then run the rooter-optimized version of lxc-wheezy: mkdir -p /data/wheezy mkdir -p /data/wheezy/mnt ./lxc-wheezy -p /data/wheezy Edit /data/wheezy/config and add (or edit) two lines: lxc.utsname = wheezy lxc.mount.entry=/data/pub /data/wheezy/rootfs/pub none bind 0 0 Might need to change SSH listen port: vi /data/wheezy/rootfs/etc/ssh/sshd_config # edit "Port" line Try starting lxc-start: lxc-start -n wheezy -f /data/wheezy/config If you are SSH'd in and get an error about file descriptor fd 7, you may need to do: lxc-start -n wheezy -f /data/wheezy/config 7<&- 8<&- Horray! ## Blueprints git clone /data/blueprints/ /data/social/rootfs/root/.blueprint --bare -b rooter-wheezy ## TODOs create a set of rooter-x86-net6501 images with: - set local subnet to 192.168.42.1 - overlay devices.tar.gz (?) - default mounted /data and /cgroup (via base-files) - lxc wheezy init script, start by default ttt functionality: - *.ttt.rooter.is DNS - openvpn tunnel - remote ipv4 reverse proxy - remote netcat ssh proxy/tunnel - some kind of media sharing - local hostname DNS TODO: remove excess kernel modules for faster/cleaner boot NOTE: dd if=openwrt-x86-net6501-rootfs-squashfs.img of=/dev/sda2 bs=1M - move ./files content to packages - clean up and upstream net6501 directions, notes, patches - clean up and upstream libattr and lxc stuff patches TODO: zgrep, IKCONFIG_PROC TODO: mtr broken? ### CONFIG BACKUP /data/wheezy/conf: lxc.utsname = ttt.rooter.is lxc.tty = 4 lxc.pts = 1024 lxc.rootfs = /data/wheezy/rootfs lxc.rootfs.mount = /data/wheezy/mnt lxc.cgroup.devices.deny = a # /dev/null and zero lxc.cgroup.devices.allow = c 1:3 rwm lxc.cgroup.devices.allow = c 1:5 rwm # consoles lxc.cgroup.devices.allow = c 5:1 rwm lxc.cgroup.devices.allow = c 5:0 rwm lxc.cgroup.devices.allow = c 4:0 rwm lxc.cgroup.devices.allow = c 4:1 rwm # /dev/{,u}random lxc.cgroup.devices.allow = c 1:9 rwm lxc.cgroup.devices.allow = c 1:8 rwm lxc.cgroup.devices.allow = c 136:* rwm lxc.cgroup.devices.allow = c 5:2 rwm # rtc lxc.cgroup.devices.allow = c 254:0 rwm # mounts point lxc.mount.entry=proc /data/wheezy/rootfs/proc proc nodev,noexec,nosuid 0 0 lxc.mount.entry=sysfs /data/wheezy/rootfs/sys sysfs defaults 0 0 lxc.mount.entry=/data/pub /data/wheezy/rootfs/pub none bind 0 0 ### Wireless Want at least: wpa-supplicant wpa-cli hostapd-mini wireless-tools collectd-mod-wireless iw iwconfig kmod-mac80211 kmod-cfg80211 crda iwinfo libiwinfo-lua Possibly want: wavemon (monitoring) TODO: configure open throttled guest WLAN: http://wiki.openwrt.org/doc/recipes/guest-wlan#step.5limit.bandwidth.of.the.connection