Difference between revisions of "Setup MediaWiki on AWS"

From dftwiki3
Jump to: navigation, search
 
(14 intermediate revisions by the same user not shown)
Line 11: Line 11:
 
|}
 
|}
 
<br />
 
<br />
 +
<br />
 +
<onlydft>
 +
=Resources=
 +
 +
* [[Media:dftwiki_to_EC2_diagrams.key.zip]]
 +
 +
=Organization=
 +
[[Image:dftwiki_to_EC2.png|center|700px]]
 +
<br />
 +
[[Image:dftwiki_to_EC2_2.png|center|700px]]
 +
<br />
 +
[[Image:dftwiki_to_EC2_3.png|center|700px]]
 +
 
<br />
 
<br />
 
=Mediawiki Installation/Upgrade=
 
=Mediawiki Installation/Upgrade=
Line 24: Line 37:
 
==Updated Extensions==
 
==Updated Extensions==
 
I had to update several extensions to match the new format.
 
I had to update several extensions to match the new format.
−
<onlydft>
+
 
 
<source lang="php">
 
<source lang="php">
 
<?php
 
<?php
Line 63: Line 76:
  
 
</source>
 
</source>
−
</onlydft>
+
 
 
=AWS=
 
=AWS=
 
==EC2==
 
==EC2==
 
* Go for free-tier t2-micro with attached EBS drive (20GB)
 
* Go for free-tier t2-micro with attached EBS drive (20GB)
 
* Set OS to Ubuntu Server 16.04.   
 
* Set OS to Ubuntu Server 16.04.   
−
* <onlydft>EC2 Insance i-0ad8bd01648543063</onlydft>
+
* EC2 Insance i-0ad8bd01648543063
 
* Install MediaWiki on it, without data
 
* Install MediaWiki on it, without data
 
* Install emacs
 
* Install emacs
 
* Make an AMI out of it
 
* Make an AMI out of it
 
* Move the images, extensions, skins, etc to newly installed MW.
 
* Move the images, extensions, skins, etc to newly installed MW.
−
* Install ddclient.  <onlydft>Map domain to hadoop02.dyndns.org</onlydft>
+
* Install ddclient.  Map domain to hadoop02.dyndns.org
  
 
==S3==
 
==S3==
−
* Create a bucket for backups <onlydft>(backup-dft-hadoop02)</onlydft>
+
* Create a bucket for backups (backup-dft-hadoop02)
  
 
==Backups==
 
==Backups==
Line 87: Line 100:
 
* All information in /home/ubuntu/.duply/test/conf
 
* All information in /home/ubuntu/.duply/test/conf
 
* Duplicity conf file:
 
* Duplicity conf file:
−
<onlydft>
+
 
 
::<source lang="text">
 
::<source lang="text">
 
GPG_KEY='88E405A1'
 
GPG_KEY='88E405A1'
Line 97: Line 110:
 
MAX_AGE=1M
 
MAX_AGE=1M
 
</source>
 
</source>
−
</onlydft>
+
 
 
* create script in /usr/local/bin, called backupDftWiki3MySql.sh to do mysqldump of MW to an sql file, and store it:
 
* create script in /usr/local/bin, called backupDftWiki3MySql.sh to do mysqldump of MW to an sql file, and store it:
 
<source lang="bash">
 
<source lang="bash">
Line 109: Line 122:
 
* Created new domain dominiquefthiebaut.com
 
* Created new domain dominiquefthiebaut.com
 
* Create record to map it to EC2 instance.
 
* Create record to map it to EC2 instance.
−
* Find public IP of EC2 <onlydft>35.153.83.30</onlydft> and put in edit-box in record.  Accept defaults.
+
* Find public IP of EC2 35.153.83.30 and put in edit-box in record.  Accept defaults.
 
* Disable ddclient on EC2 and with dyndns.
 
* Disable ddclient on EC2 and with dyndns.
 
* Change the name of the host in /etc/hosts from hadoop02 to new hostname.
 
* Change the name of the host in /etc/hosts from hadoop02 to new hostname.
Line 130: Line 143:
  
 
=Script to Copy MediaWiki from Xgridmac2 to EC2=
 
=Script to Copy MediaWiki from Xgridmac2 to EC2=
−
<onlydft>
+
 
 
==On Xgridmac2==
 
==On Xgridmac2==
 
::<source lang="bash">
 
::<source lang="bash">
Line 141: Line 154:
 
# backup dftwiki3 database to local zipped sql file on cirrus
 
# backup dftwiki3 database to local zipped sql file on cirrus
 
cd ~/dftwiki3
 
cd ~/dftwiki3
−
ssh -t  dthiebaut@www.science.smith.edu '( mysqldump -u thiebaut -pdft45tfd dftwiki3 )' > dftwiki3.sql 2> /dev/null
+
ssh -t  dthiebaut@www.science.smith.edu '( mysqldump -u thiebaut -ptaratata dftwiki3 )' > dftwiki3.sql 2> /dev/null
 
gzip -f dftwiki3.sql
 
gzip -f dftwiki3.sql
  
Line 159: Line 172:
 
# EC2 instance
 
# EC2 instance
 
# -------------------------------------------------------------------------
 
# -------------------------------------------------------------------------
−
gunzip -c  dftwiki3.sql.gz |ssh -t ubuntu@dominiquefthiebaut.com "cat - | mysql -u thiebaut -pdft45tfd dftwiki3 " 2> /dev/null
+
gunzip -c  dftwiki3.sql.gz |ssh -t ubuntu@dominiquefthiebaut.com "cat - | mysql -u thiebaut -ptaratata dftwiki3 " 2> /dev/null
  
 
# -------------------------------------------------------------------------
 
# -------------------------------------------------------------------------
Line 175: Line 188:
 
</source>
 
</source>
  
−
</onlydft>
+
 
 
<br />
 
<br />
−
=Raw Log=
+
=Log of Transfer to Virtual Host=
−
<onlydft>
 
−
1/4/18
 
  
−
upgrade dftwiki
+
== Current versions on wiki special page==
  
−
+——————————————————————————————+
+
* http://www.science.smith.edu/dftwiki/index.php/Special:Version
−
Current versions on wiki special page
+
* Product Version
−
+——————————————————————————————+
+
* MediaWiki 1.20.5
 +
* PHP 5.5.9-1ubuntu4.20 (apache2handler)
 +
* MySQL 5.5.53-0ubuntu0.14.04.1
  
−
http://www.science.smith.edu/dftwiki/index.php/Special:Version
+
==  dftwiki/includes/DefaultSettings.php ==
−
Product Version
 
−
MediaWiki 1.20.5
 
−
PHP 5.5.9-1ubuntu4.20 (apache2handler)
 
−
MySQL 5.5.53-0ubuntu0.14.04.1
 
  
−
+——————————————————————————————+
+
/** Database host name or IP address */
−
  dftwiki/includes/DefaultSettings.php
+
$wgDBserver = 'localhost';
−
+——————————————————————————————+
+
/** Database port number (for PostgreSQL) */
 +
$wgDBport = 5432;
 +
/** Name of the database */
 +
$wgDBname = 'my_wiki';
 +
  /** Database username */
 +
$wgDBuser = 'wikiuser';
 +
/** Database user's password */
 +
$wgDBpassword = '';
 +
/** Database type */
 +
$wgDBtype = 'mysql';
 +
/** Whether to use SSL in DB connection. */
 +
$wgDBssl = false;
 +
/** Whether to use compression in DB connection. */
 +
$wgDBcompress = false;
  
−
/** Database host name or IP address */
 
−
$wgDBserver = 'localhost';
 
−
/** Database port number (for PostgreSQL) */
 
−
$wgDBport = 5432;
 
−
/** Name of the database */
 
−
$wgDBname = 'my_wiki';
 
−
/** Database username */
 
−
$wgDBuser = 'wikiuser';
 
−
/** Database user's password */
 
−
$wgDBpassword = '';
 
−
/** Database type */
 
−
$wgDBtype = 'mysql';
 
−
/** Whether to use SSL in DB connection. */
 
−
$wgDBssl = false;
 
−
/** Whether to use compression in DB connection. */
 
−
$wgDBcompress = false;
 
  
−
+——————————————————————————————+
+
== dftwiki/LocalSettings.php ==
−
dftwiki/LocalSettings.php
 
−
+——————————————————————————————+
 
  
−
## Database settings
 
−
$wgDBtype          = "mysql";
 
−
# change to localhost on 7/26/11
 
−
#$wgDBserver        = "maven.smith.edu";
 
−
#$wgDBserver        = "scinix.smith.edu";
 
−
$wgDBserver        = "localhost";
 
−
$wgDBname          = "dftwiki";
 
−
$wgDBuser          = "thiebaut";
 
−
$wgDBpassword      = "tataratata";
 
  
 +
## Database settings
 +
$wgDBtype          = "mysql";
 +
# change to localhost on 7/26/11
 +
#$wgDBserver        = "maven.smith.edu";
 +
#$wgDBserver        = "scinix.smith.edu";
 +
$wgDBserver        = "localhost";
 +
$wgDBname          = "dftwiki";
 +
$wgDBuser          = "thiebaut";
 +
$wgDBpassword      = "tataratata";
  
−
+——————————————————————————————+
 
−
Backup dftwiki on cirrus
 
−
+——————————————————————————————+
 
  
−
xgridmac2
+
== Backup dftwiki on cirrus ==
−
ssh -Y dthiebaut@www.science.smith.edu
 
−
cd /var/www/departments/cs/dftwiki
 
  
−
cd /var/www/departments/cs
+
* on xgridmac2
−
tar --ignore-failed-read -czf dftwiki_010418.tgz dftwiki/
+
ssh -Y dthiebaut@www.science.smith.edu
 +
cd /var/www/departments/cs/dftwiki
 +
 +
cd /var/www/departments/cs
 +
tar --ignore-failed-read -czf dftwiki_010418.tgz dftwiki/
 +
 +
cd /var/www/departments/cs
 +
mysqldump --user=thiebaut --password=taratata dftwiki > dftwiki_010418.sql
 +
tar -czvf dftwiki_010418.sql.tgz dftwiki_010418.sql
 +
  
−
cd /var/www/departments/cs
+
* rsync tgz files to xgridmac2, then to hadoop0:temp/
−
mysqldump --user=thiebaut --password=dft45tfd dftwiki > dftwiki_010418.sql
 
−
tar -czvf dftwiki_010418.sql.tgz dftwiki_010418.sql
 
  
 +
== Add VM with Ubuntu server on hadoop0 ==
  
−
rsync tgz files to xgridmac2, then to hadoop0:temp/
+
== Run  VirtualBox on hadoop0 to create hadoop01 ==
  
−
+——————————————————————————————+
+
== Once on Hadoop01 ==
−
Add VM with Ubuntu server on hadoop0
 
−
+——————————————————————————————+
 
  
−
+——————————————————————————————+
+
* Install ddclient
−
Run  VirtualBox on hadoop0 to create hadoop01)
+
# Configuration file for ddclient generated by debconf
−
+——————————————————————————————+
+
#
−
 
+
# /etc/ddclient.conf  
−
+——————————————————————————————+
+
−
Once on Hadoop01
+
protocol=dyndns2
−
+——————————————————————————————+
+
use=web, web=checkip.dyndns.com, web-skip='IP Address'
−
Install ddclient
+
server=members.dyndns.org
−
# Configuration file for ddclient generated by debconf
+
login=dthiebaut
−
#
+
password='super##man'
−
# /etc/ddclient.conf
+
hadoop01.dyndns.org
−
 
+
−
protocol=dyndns2
+
* sudo apt-get install emacs
−
use=web, web=checkip.dyndns.com, web-skip='IP Address'
+
* set network to bridge
−
server=members.dyndns.org
+
* reset hadoop01
−
login=dthiebaut
+
* edit /etc/apache2/mysql/mysqld…/*.conf and replace 127.0.0.1 by 0.0.0.0 to allow access from outside
−
password='tom01li'
+
* on hadoop01
−
hadoop01.dyndns.org
+
 +
mysql -u root -p
 +
 +
* create user dftwiki with password taratata
  
−
sudo apt-get install emacs
+
     create user ‘thiebaut’@‘localhost’ identified by ‘taratata’;
−
 
 
−
set network to bridge
 
−
 
 
−
reset hadoop01
 
−
 
 
−
edit /etc/apache2/mysql/mysqld…/*.conf and replace 127.0.0.1 by 0.0.0.0 to allow access from outside
 
−
 
 
−
on hadoop01
 
−
 
 
−
mysql -u root -p
 
−
 
 
−
create user dftwiki with password dft45tfd
 
−
 
 
−
     create user ‘thiebaut’@‘localhost’ identified by ‘dft45tfd’;
 
 
     grant all privileges on * . * to ‘thiebaut’@‘localhost';
 
     grant all privileges on * . * to ‘thiebaut’@‘localhost';
  
Line 291: Line 285:
 
      
 
      
  
−
at bash prompt:
+
* on bash prompt  
 +
* add “USE dftwiki;” as first line of mysqldump sql file
 +
* echo “USE dftwiki;” > newfile.sql
 +
* cat dftwiki_010418.sql >> newfile.sql
 +
* mv newfile.sql dftwiki_010418.sql
  
−
add “USE dftwiki;” as first line of mysqldump sql file
+
== restore database ==
  
−
echo “USE dftwiki;” > newfile.sql
+
* mysql -u thiebaut -p < dftwiki_010418.sql
−
cat dftwiki_010418.sql >> newfile.sql
 
−
mv newfile.sql dftwiki_010418.sql
 
  
−
restore database
+
::(takes a long time)
  
−
mysql -u thiebaut -p < dftwiki_010418.sql
+
*  sudo apt-get install php5.6-gd php5.6-mysql php5.6-dom php5.6-cli php5.6-json php5.6-common php5.6-mbstring php5.6-opcache php5.6-readline
  
−
(takes a long time)
+
* download mediawiki-1.27.4
  
−
sudo apt-get install php5.6-gd php5.6-mysql php5.6-dom php5.6-cli php5.6-json php5.6-common php5.6-mbstring php5.6-opcache php5.6-readline
 
  
−
download mediawiki-1.27.4
 
  
 +
==dftwiki==
  
−
+——————————————————————————————+
+
* dftwiki works
−
dftwiki
 
−
+——————————————————————————————+
 
−
dftwiki works
 
  
−
+——————————————————————————————+
+
==dftwiki2==
−
dftwiki2
 
−
+——————————————————————————————+
 
  
−
Download media wiki 1.27
+
* Download media wiki 1.27
 +
* Install new skin: Chameleon:
 +
* Install composer
 +
* do installation of chameleon skin
 +
* edit composer.json and add
 +
 +
    "mediawiki/chameleon-skin": "~1.0"
 +
 +
:: in composer.json, in require section.  Don’t forget comma before that line
  
−
Install new skin: Chameleon:
+
* composer update "mediawiki/chameleon-skin"
  
−
Install composer
+
* Remove discussion tab:
−
do installation of chameleon skin
 
−
edit composer.json and add
 
  
−
  "mediawiki/chameleon-skin": "~1.0"
+
* type “MediaWiki:Common.css” in wiki search box, then edit window and add
 +
 +
#ca-talk {
 +
    display: none !important;
 +
}
 +
 +
:: to it.
  
−
in composer.json, in require section.  Don’t forget comma before that line
+
* Updated onlydft, and only smith
  
−
composer update "mediawiki/chameleon-skin"
+
* Meta Tag extension not working…  Need to put into a class?
  
−
Remove discussion tab:
+
== upgrade to MW 1.30 ==
  
−
type “MediaWiki:Common.css” in wiki search box, then edit window and add
+
* make dominique use www-data as default group
 +
 +
sudo usermod -g www-data dominique
 +
 +
* backup mysql database and directory tree
 +
* for mysql backup dftwiki2 from hadoop0 into a sql file
 +
* for directory tree, just create a zip file and rsync it somewhere else.
  
−
#ca-talk {
+
* create new database and copy sql backup into it.
−
    display: none !important;
 
−
}
 
  
−
to it.
+
* download mediawiki 1.30
 +
* untar
 +
* put in /var/www/html/dftwiki3
  
−
Updated onlydft, and only smith
+
* point browser to hadoop01.dyndns.org/mw-config
 +
* give info on database containing copy of working database
 +
 
 +
* Create chameleon skin:
  
−
Meta Tag extension not working… Need to put into a class?
+
* emacs -nw composer.json , and add  "mediawiki/chameleon-skin": "~1.0" in the “require”
 +
section
  
−
+——————————————————————————————+
+
* sudo composer update "mediawiki/chameleon-skin"
−
upgrade to MW 1.30
 
−
+——————————————————————————————+
 
  
−
make dominique use www-data as default group
+
* import/rsync one extension at a time into LocalSettings.php
  
−
sudo usermod -g www-data dominique
+
* Pick chameleon and navhead as the default font.
  
−
backup mysql database and directory tree
+
</source>
−
for mysql backup dftwiki2 from hadoop0 into a sql file
 
−
for directory tree, just create a zip file and rsync it somewhere else.
 
  
−
create new database and copy sql backup into it.
+
=Install LAMP on Ubuntu EC2=
  
−
download mediawiki 1.30
 
−
untar
 
−
put in /var/www/html/dftwiki3
 
  
−
point browser to hadoop01.dyndns.org/mw-config
+
== Install LAMP on Ubuntu on AWS==
−
give info on database containing copy of working database
 
−
 
 
−
Create chameleon skin:
 
  
−
  emacs -nw composer.json , and add "mediawiki/chameleon-skin": "~1.0" in the “require”
+
* Pick free-tier
−
  section
+
* Ubuntu server
−
 
+
−
  sudo composer update "mediawiki/chameleon-skin"
+
sudo apt-get update
−
 
+
sudo apt-get dist-upgrade
−
import/rsync one extension at a time into LocalSettings.php
+
sudo apt-get install apache2
−
 
+
  sudo apt-get install emacs
−
Pick chameleon and navhead as the default font.
+
sudo a2enmod rewrite
 +
sudo apt-get install php libapache2-mod-php php-mcrypt
 +
ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'
 +
sudo apt-get install mysql-server
 +
sudo adduser ubuntu www-data
 +
  sudo emacs /etc/hosts -nw
 +
 +
cat /etc/hosts
 +
 +
127.0.0.1 localhost
 +
127.0.0.1 ip-172-30-0-30
 +
   
 +
# The following lines are desirable for IPv6 capable hosts
 +
  ::1 ip6-localhost ip6-loopback
 +
fe00::0 ip6-localnet
 +
ff00::0 ip6-mcastprefix
 +
ff02::1 ip6-allnodes
 +
ff02::2 ip6-allrouters
 +
ff02::3 ip6-allhosts
  
 +
 +
=Setup S3cmd on hadoop0=
  
 +
* install s3cmd
 +
* s3cmd --configure
 +
::* Access Key: AKIAINNPVCF6X257DRPQ
 +
::* Secret Key: GE06l3VK1QHpxFB5Arywu6MJ3xaeWlNfDzoXdyns
 +
::* encryption key: {eZDfswapkcoTXZoKBxLKw2ZmNNwiTKb
 +
* works
 +
* for first backup, go to hadoop0, in 1TB, create s3 dir, then backup-dft-hadoop02 subdir, cd there and issue command:
 +
 +
s3cmd sync -r s3://backup-dft-hadoop02 .
 +
 +
: and it backs s3 down.
 +
* created crontab entry on hadoop0:
 +
 +
0 5 * * 0 /usr/bin/s3cmd sync -q -r s3://backup-dft-hadoop02/ /mnt/1TB/s3/backup-dft-hadoop02/
 +
  
 
</onlydft>
 
</onlydft>

Latest revision as of 18:42, 24 January 2018

D. Thiebaut (talk) 16:44, 13 January 2018 (EST)


This page contains very sketchy steps for setting up (in my case moving) an existing mediawiki installation to AWS. This list is mostly for myself to remember what steps I have taken.

MediaWiki.png




...







Retrieved from "http://dominiquethiebaut.com/dftwiki3/index.php?title=Setup_MediaWiki_on_AWS&oldid=32571"