Difference between revisions of "Setup MediaWiki on AWS"

From dftwiki3
Jump to: navigation, search
 
(12 intermediate revisions by the same user not shown)
Line 11: Line 11:
 
|}
 
|}
 
<br />
 
<br />
 +
<br />
 +
<onlydft>
 +
=Resources=
 +
 +
* [[Media:dftwiki_to_EC2_diagrams.key.zip]]
 +
 +
=Organization=
 +
[[Image:dftwiki_to_EC2.png|center|700px]]
 +
<br />
 +
[[Image:dftwiki_to_EC2_2.png|center|700px]]
 +
<br />
 +
[[Image:dftwiki_to_EC2_3.png|center|700px]]
 +
 
<br />
 
<br />
 
=Mediawiki Installation/Upgrade=
 
=Mediawiki Installation/Upgrade=
Line 24: Line 37:
 
==Updated Extensions==
 
==Updated Extensions==
 
I had to update several extensions to match the new format.
 
I had to update several extensions to match the new format.
−
<onlydft>
+
 
 
<source lang="php">
 
<source lang="php">
 
<?php
 
<?php
Line 63: Line 76:
  
 
</source>
 
</source>
−
</onlydft>
+
 
 
=AWS=
 
=AWS=
 
==EC2==
 
==EC2==
 
* Go for free-tier t2-micro with attached EBS drive (20GB)
 
* Go for free-tier t2-micro with attached EBS drive (20GB)
 
* Set OS to Ubuntu Server 16.04.   
 
* Set OS to Ubuntu Server 16.04.   
−
* <onlydft>EC2 Insance i-0ad8bd01648543063</onlydft>
+
* EC2 Insance i-0ad8bd01648543063
 
* Install MediaWiki on it, without data
 
* Install MediaWiki on it, without data
 
* Install emacs
 
* Install emacs
 
* Make an AMI out of it
 
* Make an AMI out of it
 
* Move the images, extensions, skins, etc to newly installed MW.
 
* Move the images, extensions, skins, etc to newly installed MW.
−
* Install ddclient.  <onlydft>Map domain to hadoop02.dyndns.org</onlydft>
+
* Install ddclient.  Map domain to hadoop02.dyndns.org
  
 
==S3==
 
==S3==
−
* Create a bucket for backups <onlydft>(backup-dft-hadoop02)</onlydft>
+
* Create a bucket for backups (backup-dft-hadoop02)
  
 
==Backups==
 
==Backups==
Line 87: Line 100:
 
* All information in /home/ubuntu/.duply/test/conf
 
* All information in /home/ubuntu/.duply/test/conf
 
* Duplicity conf file:
 
* Duplicity conf file:
−
<onlydft>
+
 
 
::<source lang="text">
 
::<source lang="text">
 
GPG_KEY='88E405A1'
 
GPG_KEY='88E405A1'
Line 97: Line 110:
 
MAX_AGE=1M
 
MAX_AGE=1M
 
</source>
 
</source>
−
</onlydft>
+
 
 
* create script in /usr/local/bin, called backupDftWiki3MySql.sh to do mysqldump of MW to an sql file, and store it:
 
* create script in /usr/local/bin, called backupDftWiki3MySql.sh to do mysqldump of MW to an sql file, and store it:
 
<source lang="bash">
 
<source lang="bash">
Line 109: Line 122:
 
* Created new domain dominiquefthiebaut.com
 
* Created new domain dominiquefthiebaut.com
 
* Create record to map it to EC2 instance.
 
* Create record to map it to EC2 instance.
−
* Find public IP of EC2 <onlydft>35.153.83.30</onlydft> and put in edit-box in record.  Accept defaults.
+
* Find public IP of EC2 35.153.83.30 and put in edit-box in record.  Accept defaults.
 
* Disable ddclient on EC2 and with dyndns.
 
* Disable ddclient on EC2 and with dyndns.
 
* Change the name of the host in /etc/hosts from hadoop02 to new hostname.
 
* Change the name of the host in /etc/hosts from hadoop02 to new hostname.
Line 130: Line 143:
  
 
=Script to Copy MediaWiki from Xgridmac2 to EC2=
 
=Script to Copy MediaWiki from Xgridmac2 to EC2=
−
<onlydft>
+
 
 
==On Xgridmac2==
 
==On Xgridmac2==
 
::<source lang="bash">
 
::<source lang="bash">
Line 141: Line 154:
 
# backup dftwiki3 database to local zipped sql file on cirrus
 
# backup dftwiki3 database to local zipped sql file on cirrus
 
cd ~/dftwiki3
 
cd ~/dftwiki3
−
ssh -t  dthiebaut@www.science.smith.edu '( mysqldump -u thiebaut -pdft45tfd dftwiki3 )' > dftwiki3.sql 2> /dev/null
+
ssh -t  dthiebaut@www.science.smith.edu '( mysqldump -u thiebaut -ptaratata dftwiki3 )' > dftwiki3.sql 2> /dev/null
 
gzip -f dftwiki3.sql
 
gzip -f dftwiki3.sql
  
Line 159: Line 172:
 
# EC2 instance
 
# EC2 instance
 
# -------------------------------------------------------------------------
 
# -------------------------------------------------------------------------
−
gunzip -c  dftwiki3.sql.gz |ssh -t ubuntu@dominiquefthiebaut.com "cat - | mysql -u thiebaut -pdft45tfd dftwiki3 " 2> /dev/null
+
gunzip -c  dftwiki3.sql.gz |ssh -t ubuntu@dominiquefthiebaut.com "cat - | mysql -u thiebaut -ptaratata dftwiki3 " 2> /dev/null
  
 
# -------------------------------------------------------------------------
 
# -------------------------------------------------------------------------
Line 175: Line 188:
 
</source>
 
</source>
  
−
</onlydft>
+
 
 
<br />
 
<br />
−
=Raw Log=
+
=Log of Transfer to Virtual Host=
−
<onlydft>
 
−
<source lang="text">
 
−
1/4/18
 
  
−
upgrade dftwiki
+
== Current versions on wiki special page==
  
−
+——————————————————————————————+
+
* http://www.science.smith.edu/dftwiki/index.php/Special:Version
−
Current versions on wiki special page
+
* Product Version
−
+——————————————————————————————+
+
* MediaWiki 1.20.5
 +
* PHP 5.5.9-1ubuntu4.20 (apache2handler)
 +
* MySQL 5.5.53-0ubuntu0.14.04.1
  
−
http://www.science.smith.edu/dftwiki/index.php/Special:Version
+
==  dftwiki/includes/DefaultSettings.php ==
−
Product Version
 
−
MediaWiki 1.20.5
 
−
PHP 5.5.9-1ubuntu4.20 (apache2handler)
 
−
MySQL 5.5.53-0ubuntu0.14.04.1
 
  
−
+——————————————————————————————+
+
/** Database host name or IP address */
−
  dftwiki/includes/DefaultSettings.php
+
$wgDBserver = 'localhost';
−
+——————————————————————————————+
+
/** Database port number (for PostgreSQL) */
 +
$wgDBport = 5432;
 +
/** Name of the database */
 +
$wgDBname = 'my_wiki';
 +
  /** Database username */
 +
$wgDBuser = 'wikiuser';
 +
/** Database user's password */
 +
$wgDBpassword = '';
 +
/** Database type */
 +
$wgDBtype = 'mysql';
 +
/** Whether to use SSL in DB connection. */
 +
$wgDBssl = false;
 +
/** Whether to use compression in DB connection. */
 +
$wgDBcompress = false;
  
−
/** Database host name or IP address */
 
−
$wgDBserver = 'localhost';
 
−
/** Database port number (for PostgreSQL) */
 
−
$wgDBport = 5432;
 
−
/** Name of the database */
 
−
$wgDBname = 'my_wiki';
 
−
/** Database username */
 
−
$wgDBuser = 'wikiuser';
 
−
/** Database user's password */
 
−
$wgDBpassword = '';
 
−
/** Database type */
 
−
$wgDBtype = 'mysql';
 
−
/** Whether to use SSL in DB connection. */
 
−
$wgDBssl = false;
 
−
/** Whether to use compression in DB connection. */
 
−
$wgDBcompress = false;
 
  
−
+——————————————————————————————+
+
== dftwiki/LocalSettings.php ==
−
dftwiki/LocalSettings.php
 
−
+——————————————————————————————+
 
  
−
## Database settings
 
−
$wgDBtype          = "mysql";
 
−
# change to localhost on 7/26/11
 
−
#$wgDBserver        = "maven.smith.edu";
 
−
#$wgDBserver        = "scinix.smith.edu";
 
−
$wgDBserver        = "localhost";
 
−
$wgDBname          = "dftwiki";
 
−
$wgDBuser          = "thiebaut";
 
−
$wgDBpassword      = "tataratata";
 
  
 +
## Database settings
 +
$wgDBtype          = "mysql";
 +
# change to localhost on 7/26/11
 +
#$wgDBserver        = "maven.smith.edu";
 +
#$wgDBserver        = "scinix.smith.edu";
 +
$wgDBserver        = "localhost";
 +
$wgDBname          = "dftwiki";
 +
$wgDBuser          = "thiebaut";
 +
$wgDBpassword      = "tataratata";
  
−
+——————————————————————————————+
 
−
Backup dftwiki on cirrus
 
−
+——————————————————————————————+
 
  
−
xgridmac2
+
== Backup dftwiki on cirrus ==
−
ssh -Y dthiebaut@www.science.smith.edu
 
−
cd /var/www/departments/cs/dftwiki
 
  
−
cd /var/www/departments/cs
+
* on xgridmac2
−
tar --ignore-failed-read -czf dftwiki_010418.tgz dftwiki/
+
ssh -Y dthiebaut@www.science.smith.edu
 +
cd /var/www/departments/cs/dftwiki
 +
 +
cd /var/www/departments/cs
 +
tar --ignore-failed-read -czf dftwiki_010418.tgz dftwiki/
 +
 +
cd /var/www/departments/cs
 +
mysqldump --user=thiebaut --password=taratata dftwiki > dftwiki_010418.sql
 +
tar -czvf dftwiki_010418.sql.tgz dftwiki_010418.sql
 +
  
−
cd /var/www/departments/cs
+
* rsync tgz files to xgridmac2, then to hadoop0:temp/
−
mysqldump --user=thiebaut --password=dft45tfd dftwiki > dftwiki_010418.sql
 
−
tar -czvf dftwiki_010418.sql.tgz dftwiki_010418.sql
 
  
 +
== Add VM with Ubuntu server on hadoop0 ==
  
−
rsync tgz files to xgridmac2, then to hadoop0:temp/
+
== Run  VirtualBox on hadoop0 to create hadoop01 ==
  
−
+——————————————————————————————+
+
== Once on Hadoop01 ==
−
Add VM with Ubuntu server on hadoop0
 
−
+——————————————————————————————+
 
  
−
+——————————————————————————————+
+
* Install ddclient
−
  Run VirtualBox on hadoop0 to create hadoop01)
+
  # Configuration file for ddclient generated by debconf
−
+——————————————————————————————+
+
#
 +
# /etc/ddclient.conf
 +
 +
  protocol=dyndns2
 +
use=web, web=checkip.dyndns.com, web-skip='IP Address'
 +
server=members.dyndns.org
 +
login=dthiebaut
 +
password='super##man'
 +
hadoop01.dyndns.org
 +
 +
* sudo apt-get install emacs
 +
* set network to bridge
 +
* reset hadoop01
 +
* edit /etc/apache2/mysql/mysqld…/*.conf and replace 127.0.0.1 by 0.0.0.0 to allow access from outside
 +
* on hadoop01
 +
 +
mysql -u root -p
 +
 +
* create user dftwiki with password taratata
  
−
+——————————————————————————————+
+
     create user ‘thiebaut’@‘localhost’ identified by ‘taratata’;
−
Once on Hadoop01
 
−
+——————————————————————————————+
 
−
Install ddclient
 
−
# Configuration file for ddclient generated by debconf
 
−
#
 
−
# /etc/ddclient.conf
 
−
 
 
−
protocol=dyndns2
 
−
use=web, web=checkip.dyndns.com, web-skip='IP Address'
 
−
server=members.dyndns.org
 
−
login=dthiebaut
 
−
password='super##man'
 
−
hadoop01.dyndns.org
 
−
 
 
−
sudo apt-get install emacs
 
−
 
 
−
set network to bridge
 
−
 
 
−
reset hadoop01
 
−
 
 
−
edit /etc/apache2/mysql/mysqld…/*.conf and replace 127.0.0.1 by 0.0.0.0 to allow access from outside
 
−
 
 
−
on hadoop01
 
−
 
 
−
mysql -u root -p
 
−
 
 
−
create user dftwiki with password dft45tfd
 
−
 
 
−
     create user ‘thiebaut’@‘localhost’ identified by ‘dft45tfd’;
 
 
     grant all privileges on * . * to ‘thiebaut’@‘localhost';
 
     grant all privileges on * . * to ‘thiebaut’@‘localhost';
  
Line 292: Line 285:
 
      
 
      
  
−
at bash prompt:
+
* on bash prompt  
 +
* add “USE dftwiki;” as first line of mysqldump sql file
 +
* echo “USE dftwiki;” > newfile.sql
 +
* cat dftwiki_010418.sql >> newfile.sql
 +
* mv newfile.sql dftwiki_010418.sql
  
−
add “USE dftwiki;” as first line of mysqldump sql file
+
== restore database ==
  
−
echo “USE dftwiki;” > newfile.sql
+
* mysql -u thiebaut -p < dftwiki_010418.sql
−
cat dftwiki_010418.sql >> newfile.sql
 
−
mv newfile.sql dftwiki_010418.sql
 
  
−
restore database
+
::(takes a long time)
  
−
mysql -u thiebaut -p < dftwiki_010418.sql
+
*  sudo apt-get install php5.6-gd php5.6-mysql php5.6-dom php5.6-cli php5.6-json php5.6-common php5.6-mbstring php5.6-opcache php5.6-readline
  
−
(takes a long time)
+
* download mediawiki-1.27.4
  
−
sudo apt-get install php5.6-gd php5.6-mysql php5.6-dom php5.6-cli php5.6-json php5.6-common php5.6-mbstring php5.6-opcache php5.6-readline
 
  
−
download mediawiki-1.27.4
 
  
 +
==dftwiki==
  
−
+——————————————————————————————+
+
* dftwiki works
−
dftwiki
 
−
+——————————————————————————————+
 
−
dftwiki works
 
  
−
+——————————————————————————————+
+
==dftwiki2==
−
dftwiki2
 
−
+——————————————————————————————+
 
  
−
Download media wiki 1.27
+
* Download media wiki 1.27
 +
* Install new skin: Chameleon:
 +
* Install composer
 +
* do installation of chameleon skin
 +
* edit composer.json and add
 +
 +
    "mediawiki/chameleon-skin": "~1.0"
 +
 +
:: in composer.json, in require section.  Don’t forget comma before that line
  
−
Install new skin: Chameleon:
+
* composer update "mediawiki/chameleon-skin"
  
−
Install composer
+
* Remove discussion tab:
−
do installation of chameleon skin
 
−
edit composer.json and add
 
  
−
  "mediawiki/chameleon-skin": "~1.0"
+
* type “MediaWiki:Common.css” in wiki search box, then edit window and add
 +
 +
#ca-talk {
 +
    display: none !important;
 +
}
 +
 +
:: to it.
  
−
in composer.json, in require section.  Don’t forget comma before that line
+
* Updated onlydft, and only smith
  
−
composer update "mediawiki/chameleon-skin"
+
* Meta Tag extension not working…  Need to put into a class?
  
−
Remove discussion tab:
+
== upgrade to MW 1.30 ==
−
 
 
−
type “MediaWiki:Common.css” in wiki search box, then edit window and add
 
−
 
 
−
#ca-talk {
 
−
    display: none !important;
 
−
}
 
−
 
 
−
to it.
 
−
 
 
−
Updated onlydft, and only smith
 
−
 
 
−
Meta Tag extension not working…  Need to put into a class?
 
−
 
 
−
+——————————————————————————————+
 
−
upgrade to MW 1.30
 
−
+——————————————————————————————+
 
−
 
 
−
make dominique use www-data as default group
 
  
 +
* make dominique use www-data as default group
 +
 
sudo usermod -g www-data dominique
 
sudo usermod -g www-data dominique
 +
 +
* backup mysql database and directory tree
 +
* for mysql backup dftwiki2 from hadoop0 into a sql file
 +
* for directory tree, just create a zip file and rsync it somewhere else.
  
−
backup mysql database and directory tree
+
* create new database and copy sql backup into it.
−
for mysql backup dftwiki2 from hadoop0 into a sql file
 
−
for directory tree, just create a zip file and rsync it somewhere else.
 
−
 
 
−
create new database and copy sql backup into it.
 
  
−
download mediawiki 1.30
+
* download mediawiki 1.30
−
untar
+
* untar
−
put in /var/www/html/dftwiki3
+
* put in /var/www/html/dftwiki3
  
−
point browser to hadoop01.dyndns.org/mw-config
+
* point browser to hadoop01.dyndns.org/mw-config
−
give info on database containing copy of working database
+
* give info on database containing copy of working database
 
    
 
    
−
Create chameleon skin:
+
* Create chameleon skin:
  
−
  emacs -nw composer.json , and add  "mediawiki/chameleon-skin": "~1.0" in the “require”
+
* emacs -nw composer.json , and add  "mediawiki/chameleon-skin": "~1.0" in the “require”
 
  section
 
  section
  
−
sudo composer update "mediawiki/chameleon-skin"
+
* sudo composer update "mediawiki/chameleon-skin"
  
−
import/rsync one extension at a time into LocalSettings.php
+
* import/rsync one extension at a time into LocalSettings.php
  
−
Pick chameleon and navhead as the default font.
+
* Pick chameleon and navhead as the default font.
  
 +
</source>
  
 
=Install LAMP on Ubuntu EC2=
 
=Install LAMP on Ubuntu EC2=
  
−
<source lang="text">
 
−
Install LAMP on Ubuntu on AWS
 
  
−
Pick free-tier
+
== Install LAMP on Ubuntu on AWS==
−
Ubuntu server
 
  
 +
* Pick free-tier
 +
* Ubuntu server
 +
 
  sudo apt-get update
 
  sudo apt-get update
 
  sudo apt-get dist-upgrade
 
  sudo apt-get dist-upgrade
Line 399: Line 385:
 
  sudo adduser ubuntu www-data
 
  sudo adduser ubuntu www-data
 
  sudo emacs /etc/hosts -nw
 
  sudo emacs /etc/hosts -nw
−
 
+
 
  cat /etc/hosts
 
  cat /etc/hosts
−
 
+
 
  127.0.0.1 localhost
 
  127.0.0.1 localhost
 
  127.0.0.1 ip-172-30-0-30
 
  127.0.0.1 ip-172-30-0-30
−
 
+
 
  # The following lines are desirable for IPv6 capable hosts
 
  # The following lines are desirable for IPv6 capable hosts
 
  ::1 ip6-localhost ip6-loopback
 
  ::1 ip6-localhost ip6-loopback
Line 414: Line 400:
  
 
   
 
   
 +
=Setup S3cmd on hadoop0=
  
 +
* install s3cmd
 +
* s3cmd --configure
 +
::* Access Key: AKIAINNPVCF6X257DRPQ
 +
::* Secret Key: GE06l3VK1QHpxFB5Arywu6MJ3xaeWlNfDzoXdyns
 +
::* encryption key: {eZDfswapkcoTXZoKBxLKw2ZmNNwiTKb
 +
* works
 +
* for first backup, go to hadoop0, in 1TB, create s3 dir, then backup-dft-hadoop02 subdir, cd there and issue command:
 +
 +
s3cmd sync -r s3://backup-dft-hadoop02 .
 +
 +
: and it backs s3 down.
 +
* created crontab entry on hadoop0:
 +
 +
0 5 * * 0 /usr/bin/s3cmd sync -q -r s3://backup-dft-hadoop02/ /mnt/1TB/s3/backup-dft-hadoop02/
 +
  
−
</source>
 
 
</onlydft>
 
</onlydft>
 
<br />
 
<br />

Latest revision as of 18:42, 24 January 2018

D. Thiebaut (talk) 16:44, 13 January 2018 (EST)


This page contains very sketchy steps for setting up (in my case moving) an existing mediawiki installation to AWS. This list is mostly for myself to remember what steps I have taken.

MediaWiki.png




...







Retrieved from "http://dominiquethiebaut.com/dftwiki3/index.php?title=Setup_MediaWiki_on_AWS&oldid=32571"