Geeketeria

Programming, computers, technology.

Bash, Cygwin: Remember the CRLF

This morning I wrote a bash script to find redundant accounts from a database. Writing the actual script was easy, but I ran into a problem with my environment. You see, I use windows. So the text files created in windows have that pesky CRLF line endings. So, whenever I attempted to execute a query to mysql, I would get an error saying that there was no such database.

Since I had ran into a new line problem two days ago, I decided to switch the script format to unix. I did this by doing


:e ++ff=unix
:w

The script worked after that. Bash and mysql wanted to interpret \r. It obviously didn't know what to do, so it would break afterwards.

Windows batch: automating start of day

I read someone on twitter stating that their resolution was to automate repetitive tasks. This is not my resolution, but I thought it was an excellent idea.

One task that I do every time I turn on the computer is opening 4 essential programs. This takes a few minutes. It is boring. I need to open IE, Fireforx, pidgin, and skype. I have task buttons for IE and Fireforx, and Pigdin can be found on the top of my start menu. Skype will rise and fall depending on how recent the last update was. The last two are the reason why the process can delay a bit.

So I wrote this simple batch file to open them all. The script switches to the directory of the program and starts it in the background. In bash we would use & to push the process to the background; in windows it is using the command "start". Then I created a shortcut that I dropped on the desktop. And I was finished.

Now the whole process takes around 30 seconds with a single double click.

set originalPath= %CD%
 
cd "C:\Program Files\Mozilla Firefox"
start firefox.exe
 
cd "C:\Program Files\Internet Explorer\"
start iexplore.exe
 
cd"C:\Program Files\Pidgin"
start pidgin.exe
 
cd "C:\Program Files\Skype\Phone"
start skype.exe
 
cd %originalPath%
echo End of Script

CSS two column layout

The solution:

A container includes a left and right div with floats to the right and to the left. To clear the floats, add overflow:auto; to the container.

.container 
{
    width: 100%;
    position: relative;
    overflow: auto;
}
 
.nav-left 
{
    float: left;
    width: 18%;
    padding-left: 5px;
}
 
.content 
{
    float: right;
    width: 80%;
    position: relative;
    padding-right: 5px;
}

I first found a solution that used percentage widths instead of fixed widths
http://matthewjamestaylor.com/blog/perfect-2-column-left-menu.htm

Then I found a solution for clearing the floats
http://www.quirksmode.org/css/clearing.html

Rant:

There is a reason why so many people use tables for layout: CSS column layout is a failure. It is ridiculously complex. I have learned the CSS to get two columns over and over. Since I am not a designer, I keep forgetting these kinds of css tricks, so every so often I have to learn them again (the reason why I am writing this entry.)

Now, using tables for layout is easy. Semantically it is wrong, but for practical use it is so easy that it is hard to believe that there would be more than one or two tutorials teaching you how to do that. In fact, for many, just telling them "create a layout using a table!" would be enough of a tutorial to get a new web designer pushed in the right direction. CSS column layouts, on the other hand, are abundant. Too abundant, in fact, for a problem that already had an easy solution with tables.

CapitalCampDC: Web security

Risks

Mis-configurations
Insecure code
unsafe practices

Mitigation
audit configurations
secure code
and processes

Automated security audits and penetration testing

Ben Jeavons
@benswords

48% are XSS vulnerability

Drupalsecurityreport.org

control users with text formats

slides: http://lb.com/dscc

* Text format.

Cracking drupal.

There was nothing to prevent javascript from being run.
Text formats are there to prevent these kinds of problems
Filter HTML. It strips out javascript Changing these cause a problem.

HTML purifier module.

* Trust

Know your roles
How roles are granted
Administer permissions
Administer users
Administer filters
Administer content

Idea of least privilege

Automated configuration checks
Security review module

* Recovering from an attack
Backups of the site
make sure that you upgrade
change your passwords
audit your config and code

Backups of the site

Security updates are small

FTP
Insecure

SFTP. Better.

SSL
Firesheep
intercept information to the site

Run site on TLS/SSL

securepages & securepages_prevent-hijac

security review module

Custom code, the most vulnerability lies.
drupalsecurityreport.org

Coder module
audit and peer review
drupal.org/writing-secure-code

CapitalCampDC: Prepare to scale

William O'Connor

* The more specialization you have, the better it scales.

* Performance is how fast I can get something back to person
*Scalability is how many people you can run at the same time

*Apache
* Standard, but bloated
* A well tested solution
* Apache okay for development, but move to another

*Nginx
* lighter faster cases
* WebDav won't work
* They see a lot of performance
* It is also great as a load balancer

* use drupal 7 or Pressflow platform
Support for db replication
support reverse proxy caches
mysql optimization
php5 optimazation

Opcode caching
* sort of like having a compiled version of your application
* optimizes php components
* stores the compiled php bytecode for execution in stored memory
* Result: Smaller PHP memory footprint

Recommended:
APC
* stable
* actively supported
* used eAccelator and Xcache (not recommended)

Database cluster:
* MyISAM
* good for read
* not good for writing
* It will do a table lock rather than a row lock

Switch to InnoDB
Better crash recovery

Alternatives:
MariaDB
Drop-in replacement for MySQL
superior technology

Drizzle

Alternatives
MongoDB
* Documented-oriented DB
*Examiner
*D7 module for it
** I can replace of certain portions of those things

Cassandra
column-oriented DB
*Facebook Inbox
* Eventual consistency

Separate DB Server
* One of the fastest ways to improve scalability
* DB fights vs apache in resources

Static Caching
* To make db and server to do less
* static caching. Materials that you can return without change. Returning flat and statically

Drupal modules
* boost modules. Only for anonymous files.
*AuthCache Module
A little bit more advance. More configuration. It can work with other caching engines. Some problems when you get bigger. If that is the case, you need a bigger thing
* helper module
ajaxify regions/blocks
ESI (varnish ESI)

Static cache on the same server.
* Wasted overhead.
* port to port
* segment traffic by port from the static caching.

Drupal 7 gives you easy opportunities to use static caching

Static Caching: Varnish
https://www.varnish-cache.org/about
* Free
* It can be difficult to set it up
* vcl file is a series of subroutines

Squid
Free
not designed to do this thing, but they used it to do it

AICache
* best performance, but paid service

CND
Cache content that is static (outside of full pages)
paid solutions
can be expensive

static-caching is great for anonymous traffic

Object caching
* layer of defense for db
* It caches a full object in memory.
* It will use a hash
* is leverage in memcache or APC

APC
http://www.php.net/apc
* it is the pretty quick and easy solution
* It is a single server solution
* Not distributed

* Memcache
http://memcached.org/
* It can be used distributed
* Facebook uses it
* slower in the single server instance
* provide silos/buckets can be created for information so you can distribute information across multiple servers
* memcache should be behind the firewall
* it is a hash table
* there is a one meg limit

Object/Document-module Dbs
* MongoDB can be used similarity to Memcache
* You can get object by key and run queries on it!!!
* It is a well perfoming system

Solr
* It allows the site perform much better
* You can move it out another system
* It handles the data store
* All what it must do is feed it information

Deployment
* Jenkins

Monitoring soluitons
*Zabbix

Start with application improvements

pressflow/DY
innoDB
APC

.NET Binary serialization quick reference

To serialize:

public void Serialize(MyLittleObject mylittleObject)
{
        var stream = File.Open(path, FileMode.Create);
        var binaryFormatter = new BinaryFormatter();
        
        binaryFormatter.Serialize(stream, myLittleObject);
        stream.Close();
}

To deserialize


public MyLittleObject Deserialize()
{
            var stream = File.Open(filePath, System.IO.FileMode.Open);
            var binaryFormatter = new BinaryFormatter();

            var result = (MyLittleObject)binaryFormatter.Deserialize(stream);
        
          return result;
}

Haskell a day, day 4: Syntax in functions

Pattern matching

* It seems similar to Prolog's and Erlang's pattern matching, although you have the function declaration

I got to look into how to declare functions. I think I may trip here.

Haskell a day, day 3: Intro to lists

Intro to lists

http://learnyouahaskell.com/starting-out#an-intro-to-lists

a list
[1,2,3,4,5,6]

List concatenation. And since strings are lists too, this is also the string concatenator.
[1,2,3] ++ [4,5,6]

Shifting to a list, called CONS (probably from lisp), is faster. It uses the semicolon
5:[1,2,3,4,5]

Index is done via !!. Indices start at zero
[1,2,3,4,5] !! 3

Lists can be compared with <, <=, >, >=

And we have our friends, "head," "tail," and "last."

Other functions
length
null (checks if the list is null)
reverse
take (takes a number and a list, and then extracts those elements in the list)
drop (similar, but it drops them instead)
maximum
minimum
sum
product
replicate

Ranges
[1..20] by one
[3,5..20] by three

Functions that will create an infinite list (Is this awesome or what?)
cycle
repeat

List comprehension
[x*2 | x <- [1..10], x*2 >= 12]

These seem to be very powerful. I am sure I will have to read about them later.

Tuples

* They have a defined size
* You can have them with different types
* One uses () to surround the tuple

functions connected with tuples
fst -> it takes a pair and returns the first element
snd -> it takes a pair and returns the second one

zip -> It takes two lists and it matches them into pairs

Types

Int
Integer
Float
Double
Bool
Char

Type variables. They actually have this.

Type classes. Like OOP interfaces
Eq
Ord
Show
Read
Enum
Bounded
Num
Integral
Floating

Haskell a day, day 2: Baby functions

Notes on functions

http://learnyouahaskell.com/starting-out

* The syntax for single-line functions is
functionName param1 param2 ... paramN = expressions

* The if then else statement. Else is mandatory. Why? Every expression must return something. And an expression is code that returns a value.

>>Interesting points on function names<<
* Single quotes are allowed in the names of functions, and they signify strict version or slight variation. A strict version is a version of the function that is not lazy (interesting.)

* They cannot start with a capital letter

* for ghci, we can use let = value.

Personal note here: the difference between Haskell and GHCI must be stressed. Many tutorials gloss over the difference, and this is a cause of confusions with beginners. I remember having a problem with it the first time that I worked through a Haskell tutorial. So what is the difference? GHCI, as a interactive shell, gives you a collection of nice utilities to make your life easier. Be careful to distinguish those utilities from the actual language.

Jango, our Django magic pony that spreads cuteness

My daughter designed this one. It makes me want to learn Django :P

Syndicate content