Block all 404 errors

Recently, I found huge amount of 404 error which cause my server CPU to 100%. Most of the 404 errors are come from several IP addresses.

Firstly I installed iptables to block them out:

iptables -A INPUT -s %SOME-IP-ADDRESS% -j DROP

After doing this, to save it, you need to run

service iptables save

Later, I want to find some way to automatically block them out, I found fail2ban.

After simply install fail2ban, you need to create a file called jail.local to put settings in:

[DEFAULT]

ignoreip = 127.0.0.1/8 %SOME MORE IP IF YOU LIKE%

bantime  = 3600

[apache-script-bots]

enabled  = true

port     = http,https

logpath  = %(apache_access_log)s

maxretry = 5

This means if someone tries for 5 times 404 error, there IP address will be banned for 1 hour.

You also need to put a file called apache-script-bots.conf in filter.d folder:

[Definition]

failregex = ^ .* 404 .*$

ignoreregex =

This setting tell fail2ban to check Apache access log to find all the 404 errors.

Abandon git local branch

I've merged, I've commited changes locally, but I have not decided to push it or not, so I leave it there. After several days, I want to totally abandon my local branch.

I would do below:

1. git checkout master
   It will just tell you that your local branch is different from remote.
2. git reset --hard origin/master
   This is the real effect command, it will force your local to be exactly same as remote.

Wordpress posts show question marks

I installed Wordpress on my local server, when I moved it onto remote server, all posts show just question marks like this:

?????

What I did was: mysqldump, then mysql in.

Later after lots of search, inspired by this post:

https://mathiasbynens.be/notes/mysql-utf8mb4 

I found all I need to do is to set remote database server to use utf8mb4.

Be careful, all database, tables and columns by the newest Wordpress are already utf8mb4, so I did not touch them. Just to this to change your mysql database server to use utf8mb4. Every line is needed:

[client]
default-character-set=utf8mb4

[mysql]
default-character-set=utf8mb4

[mysqld]
character-set-client-handshake=FALSE
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci

Especially this line:
character-set-client-handshake=FALSE

I don't know why Wordpress php need this line, but it do have effect. If someone know, please let me know. Thanks!

Install Hiphop on CentOS 5.4 64

On a clean CentOS 5.4 64 environment, all you need to do is following 5 steps:

1. rpm -ivh http://epel.osuosl.org/5/x86_64/epel-release-5-4.noarch.rpm
2. rpm -ivh http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1.0-9.ius.el5.noarch.rpm
3. rpm -ivh http://pkg.tag1consulting.com/hphp/x86_64/hphp-release-1.0-2.el5.noarch.rpm
4. yum install php52
5. yum install hiphop-php

Addition:

1. mv /usr/bin/gcc44 /usr/bin/gcc
2. mv /usr/bin/g++44 /usr/bin/g++
3. cd /usr/lib64; rm -f libz.so; ln -s libz.so.1.2.3 libz.so
4. cd /lib64; rm -f libz.so; ln -s /usr/lib64/libz.so.1.2.3 libz.so

Then you can run:
hphp test.php --keep-tempdir=1 --log=3

Windows XP Quick Launch Bar Always Disappear

For some unknown reason (this reason is still unknown until now), my Windows XP quick launch bar will not be shown everytime after restart. I can right click the task bar and make it shown again, but after restart, it disappear again.

The solutions are variable, but no one can solve my issue. At last, by studying this post line by line, I got the issue solved by doing the following thing (actually just one thing):

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer]
"NoSaveSettings"=dword:00000000

This is the only useful solution, all other solutions like creating folders, setting other registry keys are all useless, and may potentially damage your system. So just use this one line and no need to try others.

SimpleTest Eclipse Plugin Issue

SimpleTest is a TDD template for PHP, I feel interest to it is because it provides a plugin for Eclipse which I use for my PHP development. There is an installation guide provided by SimpleTest official website: http://www.simpletest.org/en/extension_eclipse.html, but the problem is when I tried to follow it, there are several steps not match.

Firstly, it says you can "new" a SimpleTest project, but actually there is no SimpleTest project type for you to choose. This is actually not a big deal, you can just create a normal project, it will have no bad impact.

The most frustrating thing happen later is when you run your first test, it will complain cannot find eclipse.php. Several other users complain about this issue too, see here. To solve this issue, I searched in whole package for the file, and finally got it in a file of simpletest_php.zip, once I extracted it to a folder, and in Windows->Preferences->SimpleTest, set SimpleTest folder to that extracted folder, everything solved.

Eclipse can not debug PHP web page (session terminated)

When using Eclipse with PDT to debug PHP, it can successfully debug PHP script, but everytime when tring to debug PHP webpage, it will terminated quickly with information saying of "session terminated".

The reason is because there are 3 lines must be added in php.ini.

zend_extension_ts="C:\Program Files\PHP\ext\ZendDebugger.dll" (You need to search in your Eclipse folder to find out where the file exists and use the full path to it.)
zend_debugger.allow_hosts=127.0.0.1
zend_debugger.expose_remotely=always

After adding these 3 lines, restart Apache, then it works.

Fatal error: Call to undefined function mysql_connect()

Well, I've long time not used PHP, and now I am back.

I installed the newest Apache, the newest PHP, the newest Mysql, and everything is smooth. Seems the UIs are much better than 4 years ago. I took pretty caution to the PHP configuration, and choosed to install mysql support at beginning, so everything seems ok at first.

But I met issue soon. When I call mysql_connect, it immediately says:

Fatal error: Call to undefined function mysql_connect() ...

As I am experienced engineer, I don't worry about this, checked all website for this issue, but everyone just repeatedly saying: check your php.ini file to remove the semicolon before the line of extension=php_mysql.dll, or check your extension_dir setting to make sure it's correct, something like that, blah, blah, blah... I checked again at my php.ini setting, it's pretty correct, because when I install php I already choose to have mysql support installed, so everything already correctly setting for me. I tried many other ways, to copy php.ini file into windows folder, to copy php_mysql.dll into windows folder, and also tried windows system32 folder, and also checked the environment variable of PHPRC, the problem just still be there.

Finally I got the problem solved. Now let me tell you the truth, this IS the exactly quite important step, but the PHP installation program did not do it correctly.

Very simple, that is: to copy the file libmysql.dll from your PHP folder into Windows folder. Very simple, right? But that's it.

Now we know, there are 3 files related to mysql support in php:

1. libmysql.dll (this by default is in ext folder, you can leave it there)
2. php_mysql.dll (this be default is in your php folder, you need to manually copy it into windows folder)
3. php.ini (this by default is in your php folder, you can leave it there)

You made them in correct folder, you got it.

.NET assembly and SharePoint Workflow

As you may know, Windows SharePoint Services can only be run on Windows Server 2003, it can not be run on Windows XP. But my client side development environment is Windows XP, my Visual Studio is installed on Windows XP. I need a solution to develop workflow programs on my Windows XP instead of Windows Server 2003.

After a long time research, finally I got clear how to develop workflow in Visual Studio on a Windows XP machine.

To do such work, it's very simple, but to understand, it take some time.

Ok, now I just say the simple thing.

All you need to do is just copy the .dll files you need on your machine, then register it in .net.

1. What files you need?
Some assembly dlls. Include Microsoft.SharePoint.dll, and others.

2. Where can I get these files?
These files are in an install sharepoint machine. Directory is at c:\windows\assembly\gac_msil
In that folder, you can see a lot of directories start with Microsoft.SharePoint, copy all thoese folders to your machine.

3. How do I register them?
Open your Control Panel->Administrative Tools->Microsoft .net Framework 2.0 Configuration, then click on the "Assembly Cache", then you can register them.

Ok, now you can enjoy on it!

Force Windows XP Explorer to Sort By Name

I am an old fashioned Windows user, I don't like current Windows XP Windows Explorer sorting method, which it can "remember" your settings for each folder. Instinctly I think it will waste disk space. I always missed the old Windows sorting method that is always sort files by file name, only when I tell it to sort by date, it will sort by date, but once I close it and reopen it, I hope it still sort by name instead of sort by date. And I once and always use the "detail" view of Windows Explorer.

This works perfectly before when I choose to "Not remember settings for each folder". But one day, for some unknown reason, it just can not work. Everytime I open Windows Explorer, it always sort files by date, as I did not choose "remember" settings, it will sort by date for EVERY single folder. This is really annoying and frustrating. I tried many ways to solve this problem, I sort the folders by name, then close Windows Explorer, it has no effect, next time when I open it, it still sort by date. I even tried somebody's suggestion, that is to click Ctrl plus the close button on top right corner, still same, no effect. I tried to change the "remember" setting back and forward, hope it can remember to sort by file name, well, still same, once I enable it to "remember", it remember to sort by file name, but once I change it back to "not remember", it remember to sort by date!

I tried to think what I might do before, and I noticed everytime I open Windows Explorer, it will firstly go to the page of "My Computer" which will list all my hard disks, in this view, there is no sorting method above like other folders' detail view does. But, you still can click the above column head to sort it, I tried to click all the column headers one by one, and finally go back to sort by type, then close Windows Explorer. Guess what? Next time I open it, everything works as I wish! My computer view show hard disks together as sort by type, My documents view sort files by file name, my D drive, H drive, I drive all sort by file name now.

Seems the "sort by filename" is related to "sort by type" for My Computer view. If you have similiar issue, please try this way, and let me know your result.

How to insert a data source in SharePoint Designer 2007?

Actually, the difficulty is caused by a bug of Microsoft. Below are detail steps:

 

1. Click "Connect to a database" in "Database connections", click Configure database connection, click Use Custom String, click Edit, select "Provider for OLEDB" and type the following connection string:

 

Provider=OraOLEDB.Oracle;Data Source=wise;Persist Security Info=True;User ID=wiseq;Password=wiseq;Unicode=True

 

(Replace the parameters for Data Source, User ID and Password with your own)

 

2. Click Next and you will see an error which says "an error occurred while retrieving the list of tables from the root Database", click OK, click "Specify custom Select, Update, Insert…", click Edit Command and type the command which looks like:

 

select * from project

 

click Finish, click OK.

 

3. Then, expand the left pane of SharePoint designer, expand _catalogs--fpdatasources, you will find a new xml file whose name is the same as your connection name generated. Double click it to open it. The current string shows

 

<udc:DataSource xmlns:udc="http://schemas.microsoft.com/data/udc" MajorVersion="2" MinorVersion="0">

<udc:Name>changeora</udc:Name>

<udc:ConnectionInfo>

<DataSourceControl><CDATA><asp:SqlDataSource id="SqlDataSource1" runat="server" __designer:Provider="OraOLEDB.Oracle" __designer:customcommand="true" ProviderName="System.Data.SqlClient" __designer:customconnectionstring="true" SelectCommand="select p.deptno, p.dname from scott.dept p" ConnectionString="Data Source=wise;Persist Security Info=True;User ID=wiseq;Password=wiseq;Unicode=True;Initial Catalog=root;" /></CDATA></DataSourceControl>

</udc:ConnectionInfo>

<udc:Type MajorVersion="1" MinorVersion="0" Type="Sql"/>

</udc:DataSource>

 

You can modify the provider name and delete the Initial Catalog. The expected result looks like

 

<udc:DataSource xmlns:udc="http://schemas.microsoft.com/data/udc" MajorVersion="2" MinorVersion="0">

<udc:Name>Custom Query</udc:Name>

<udc:ConnectionInfo>

<DataSourceControl><CDATA><asp:SqlDataSource runat="server" ConnectionString="Data Source=wise;User ID=wiseq;Password=wiseq;" ProviderName="System.Data.OracleClient" ID="SqlDataSource1" SelectCommand="select p.deptno, p.dname from scott.dept p" __designer:Provider="OraOLEDB.Oracle" _designer:customcommand="true" __designer:customconnectionstring="true"></asp:SqlDataSource></CDATA></DataSourceControl>

</udc:ConnectionInfo>

<udc:Type MajorVersion="1" MinorVersion="0" Type="Sql"/>

</udc:DataSource>

 

4. Save the xml. When you click Show Data and you should able to see the data populated.

Windows Server 2003 SP1 May Cause Problems To SharePoint

Today is a very happy to me. I spent so long time, and at last solved several problems.

Q1. If I put abc.htm into some folder, then I can use IE to access it, but if I put abc.asp in, IE will always complain 404 error.
A1. It's because you need to set "Allow ASP" in the "Web Service Extensions" in IIS admin tool. By default it is turned off by SharePoint.

Q2. I put an asp page in folder, but when trying to get Request.Form, it always returns empty.
A2. It's because when you enable "Anonymous Access" in IIS, you MUST also enable "Windows Integrated Authentication", otherwise your ASP can not get form content.

Q3. This is the very hard part. After install SharePoint (WSS3), I can access http://servername on server IE, but when I access http://www.servername.com, it will ask me for password, and after enter 3 times of correct password, it will fail. But on client side, it works. Also, the reporting services in SharePoint admin behavior very strange, it can be worked on http://servername, but refuse on servername.com, also in server site collection it always cause error. And all of my previous working web services now all report "Authenticaiton fail".
A3. I tried many ways and searched for long time, and at last I searched this string in Google: ""An error occurred during logon" 537 NT AUTHORITY\SYSTEM 0xC000006D" because I found everytime I logon fail, it will show an event in Eventlog include above string. By searching that string, I found a page in Google: http://www.developmentnow.com/g/91_2005_7_0_0_560780/IIS6-NT-Authentication-fails.htm, it indirect me to here: http://support.microsoft.com/?id=896861, it says the cause is you installed Windows Server 2003 SP1:

This issue occurs if you install Microsoft Windows XP Service Pack 2 (SP2) or Microsoft Windows Server 2003 Service Pack 1 (SP1). Windows XP SP2 and Windows Server 2003 SP1 include a loopback check security feature that is designed to help prevent reflection attacks on your computer. Therefore, authentication fails if the FQDN or the custom host header that you use does not match the local computer name.

And the solution is:

Method 1: Disable the loopback check

Follow these steps:

1.	Click Start, click Run, type regedit, and then click OK.
2.	In Registry Editor, locate and then click the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3.	Right-click Lsa, point to New, and then click DWORD Value.
4.	Type DisableLoopbackCheck, and then press ENTER.
5.	Right-click DisableLoopbackCheck, and then click Modify.
6.	In the Value data box, type 1, and then click OK.
7.	Quit Registry Editor, and then restart your computer.

By following that, after restart server, seems everything works ok! I must remember this in case in future I meet same error, and it may help you too.

HELP! "Firewall has detected an application listening for incoming traffic." Event ID# 861 lsass.exe : firewall, listening, detected, application, has

HELP! "Firewall has detected an application listening for incoming traffic." Event ID# 861 lsass.exe : firewall, listening, detected, application, has: "netsh firewall add allowedprogram LSASS \ C:\WINDOWS\system32\lsass.exe"

SharePoint workflow can not start

It spend me so long time to get this problem solved.

1. I found a blog, it just says: to check the log. ok, then I go to check the log. Found it says my "Shared Services Provider" doesn't work.

2. I go to admin, create a new SSP, but it complains provision does not work, go to check log again. Found it says somebody else me is the site owner.

3. Go to command line, type stsadm -o siteowner to change it to myself. Now it complains some parameter does not work.

4. Found an article saying to add the app pool id into site admin, but it already there.

5. So I type in the below command to force my account be the farm admin: C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsadm -o updatefarmcredentials -userlogin ds\sseadmin -password password, then try again.

6. It worked!

Now if you can not start your workflow, please post comment here, and I will try my best to answer your question.

Debug in Microsoft InfoPath 2003

There is no direct way to debug in Microsoft InfoPath 2003. However, Microsoft provide a very tricky way to debug in it.

All you need to do is this:

Add a "stop" in the load script.

That's it! Then when you open the InfoPath form, you can choose which debugger you want to debug it, then you can step in, or set other breakpoint in it.

Change web site ID in IIS to solve path not found error

It cost me 2 hours to find the solution.

Situation:

SharePoint server A is also a SMS distribution server, but when SMS server B tried to push package to A, it always fail.

Process:

Check log, find it always report '/w3svc/1/root path cannot find'. But there is no this path.

After check a lot of documents, I find it's something stored for IIS as register tree. The only way to see it is by using a script called adsutil.vbs.

Compare server A and server B, I found that server A 80 port ID is not 1, but 1234567. Why? Because when we setup SharePoint server, we deleted the original server application, and created a new one on port 80, so it was assigned a random number.

So the next step is to change the number to 1.

Here is how:

CSCRIPT %SYSTEMDRIVE%\Inetpub\AdminScripts\adsutil.vbs STOP_SERVER W3SVC/1234567
CSCRIPT %SYSTEMDRIVE%\Inetpub\AdminScripts\adsutil.vbs MOVE W3SVC/1234567 W3SVC/1
CSCRIPT %SYSTEMDRIVE%\Inetpub\AdminScripts\adsutil.vbs START_SERVER W3SVC/1

Result:

After change, everything is OK!

Passing url parameters to webpart in SharePoint

Sometimes you may need to create a webpart page in SharePoint that it can use the parameters passed by url. But the "Page Viewer" webpart does not allow you to use the parameters in your webpart page url. I even tried to use SharePoint Designer 2007 to modify the page directly, but still does not work.

So I think why not directly use a "Content Editor" webpart? Because a content editor webpart allows you to embed javascript code in it. We can use an "iframe" which url is generated by javascript with url parameters from the webpart page.

Then I create a webpart page, insert in a content editor webpart, and then insert the following code in it:

<script type="text/javascript">
function getQueryStringRegExp(name)
{
var reg = new RegExp("(^|\\?|&)"+ name +"=([^&]*)(\\s|&|$)", "i");
if (reg.test(location.href)) return unescape(RegExp.$2.replace(/\+/g, " "));
return "";
}

document.write("<iframe src='http://www.yourdomain.com/index.php?test="
+ getQueryStringRegExp('test') + "' width=100% height=100%></iframe>");
</script>

The function getQueryStringRegExp is for retrieve parameter value from url, which is get from here:
http://www.codebit.cn/pub/html/javascript/tip/javascript_get_url_parameter/

Installing Microsoft SQL Server 2005 Reporting Services Add-in for Microsoft SharePoint Technologies Failure

As you know, if you want to show reports created by Microsoft SQL Server 2005 Reporting Services in a SharePoint website, you need to first download and install an add-in from Microsoft official website. However, this installation probably will fail. The symptom is you can't see the report server section as you expected. To solve this problem, you can do the following things as I did:

1. Check the Reporting Services installation log. This log is normally in the %temp% folder, but please notice, it's in an upper folder of %temp% because %temp% will direct you to a number named folder such as "1", its parent folder normally will be named as "temp", that's the folder I mentioned here.

2. In the folder I mentioned in step 1, there normally will have several files named as RS_SP_0.log or RS_SP_1.log, it depends on how many times you installed or uninstalled the add-in. Check the log file with the biggest number.

3. The first line of the log file is something like: User: SYSTEM. This specified the user used for installing this add-in. And in 7 or 8 lines below it, you could probably see a line saying "******* User does not have permission to add feature to site collection: http://yourwebsite:80". If so, this article will be useful for you.

4. Now you can open your SharePoint administrator UI, in "Application Management", choose "site collection administrators", change the web application to the one it mentioned above that has no permission, and add the user name in the first line of the log in. Then you can try to install again, check the log file again, there should have no permission error anymore, and you should be able to see the report section in your administrator UI.

That's it, hope this is useful for you!

Cover Post

This blog is created for sharing some technical knowledges that I got during daily work, and hope these knowledges may useful to other people in internet.

I am an software engineer, and always an engineer for 10+ years, in next 10+ years, I hope I still could have some time to be an engieer. Solving problems already become part of my daily life, just hope not to take too much time.

I am a Chinese people live in China, and if someone in China has good English, they may reach my blog and study some knowledge, but if they can't understand English, I am sorry, as I can write, you should can understand.

This post will be the first one and the last one in this blog talking something outside of techniche. It's also for test purpose.