Friday, November 28, 2014

Day 75, #5, Median of Two Sorted Arrays

Median of Two Sorted Arrays
There are two sorted arrays A and B of size m and n respectively. Find the median of the two sorted arrays. The overall run time complexity should be O(log (m+n)).

------------------------------------------------------------------------------------------
a special case of find kth smallest
reference:
http://leetcode.com/2011/01/find-k-th-smallest-element-in-union-of.html
http://www.programcreek.com/2012/12/leetcode-median-of-two-sorted-arrays-java/
http://blog.csdn.net/yutianzuijin/article/details/11499917
un-solved:
#1 how to determine k's value: int kA = k * lenA / (lenB + lenA)
#2 why is this inclusive: endA = kA
class Solution {
public:
    double findKth (int A[], int startA,int endA, int B[], int startB,int endB, int k) {
        int lenA = endA - startA + 1;
        int lenB = endB - startB + 1;
        if (lenA == 0) {
            return B[startB + k];
        }
        if (lenB == 0) {
            return A[startA + k];
        }
        
        if (k == 0) {
            return min(A[startA],B[startB]);
        }
        
        int kA = k * lenA / (lenB + lenA); // ???
        int kB = k - kA - 1;
        kA += startA;
        kB += startB;
        
        if (A[kA] == B[kB]) {
            return A[kA];
        }
        
        if (A[kA] > B[kB]) {
            k = k - (kB - startB + 1);
            endA = kA; // inclusive, why?
            startB = kB + 1;
            
        }else {
            k = k - (kA - startA + 1);
            startA = kA + 1; 
            endB = kB; // inclusive
        }
        
        return findKth(A,startA,endA,B,startB,endB,k);
    }

    double findMedianSortedArrays(int A[], int m, int B[], int n) {
        if ((m + n) % 2 == 1) {
            return findKth(A,0,m - 1,B,0,n - 1, (m + n) / 2);
        }else {
            return (findKth(A,0,m - 1,B,0,n - 1, (m + n) / 2) + findKth(A,0,m - 1,B,0,n - 1, (m + n) / 2 - 1)) / 2.0;
        }
    }
};
Solution #2, reference:  http://www2.myoops.org/course_material/mit/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-046JFall-2005/30C68118-E436-4FE3-8C79-6BAFBB07D935/0/ps9sol.pdf
如果A[i]是中位数,则A[i]比A里i 个数都大, 比B里(m+n)/2 - i 个数都大
j = (m+n)/2 - i - 1
B[j] < A[i] < B[j + 1]
反之,A在B[j]和B[j + 1]的左侧或者右侧
class Solution {
public:
    double findMedian (int A[],int B[],int m,int n,int left,int right) {
        if (left > right) {
            return findMedian(B,A,n,m,max(0,(m+n)/2 - m),min(n,(m+n)/2));
        }
        
        int i = (left + right) / 2;
        int j = (m + n) / 2 - i - 1;
        
        if (j >= 0 && A[i] < B[j]) {
            return findMedian(A,B,m,n,i + 1, right);
        }
        
        if (j < n - 1 && A[i] > B[j + 1]) {
            return findMedian(A,B,m,n,left,i - 1);
        }
        
        if ((m + n) % 2 == 1) {
            return A[i];
        }
        if (i > 0) {
            return (A[i] + max(B[j],A[i - 1])) / 2.0;
        }
        return (A[i] + B[j]) / 2.0;
        
    }

    double findMedianSortedArrays(int A[], int m, int B[], int n) {
        if (m > n) {
            return findMedian(A,B,m,n,max(0,(m+n)/2 - n),min(m,(m+n)/2));
        }
        return findMedian(B,A,n,m,max(0,(m+n)/2 - m),min(n,(m+n)/2));
    }
};

Another one, takes half of k at each call, key us (k + 1) / 2
#1 当A里的元素不足 k / 2 个时,可以砍掉B的前 k / 2
#2 当 A[k/2] < B[k/2], 可以砍掉A的前 k / 2
以上都可以用反证法证明

k为0-based

  1. return B[BStart + k]意思为返回第 k + 1(1-based)小的elem
  2. AK = (k + 1) / 2, 为个数,所以 k 需要 + 1
  3. AStart + AK - 1, 同样道理,AK是个数(1-based),需要转换为 0-based
  4. return findKth(,....AStart + AK), AK是要被砍掉的个数,所以不用 - 1
base condition以下,k和AK,BK不可能为0,所以需要 - 1,不然AStart永远取不到


http://www.ninechapter.com/solutions/median-of-two-sorted-arrays/
class Solution {
public:
    double findKth(int A[],int m, int AStart, int B[], int n, int BStart, int k) {
        if (AStart == m) return B[BStart + k];
        if (BStart == n) return A[AStart + k];
        
        if (k == 0) return min(A[AStart],B[BStart]); 
        
        int AKey = INT_MAX;
        int BKey = INT_MAX;
        int AK = (k + 1) / 2;
        int BK = (k + 1) / 2;
        
        if (AStart + AK - 1 < m) {
            AKey = A[AStart + AK - 1];
        }
        if (BStart + BK - 1 < n) {
            BKey = B[BStart + BK - 1];
        }
        
        if (AKey < BKey) {
            return findKth(A,m,AStart + AK,B,n,BStart,k - AK);
        }else {
            return findKth(A,m,AStart,B,n,BStart + BK,k - BK);
        }
        
    }

    double findMedianSortedArrays(int A[], int m, int B[], int n) {
        int k = n + m;
        if (k % 2 == 0) {
            return (findKth(A,m, 0, B,n, 0, k / 2 - 1) + findKth(A,m,0, B, n, 0, k / 2)) / 2.0 ;
        } else {
            return findKth(A,m,0, B, n, 0, k / 2);
        }
    }
};

Thursday, November 27, 2014

Notes: sorting


select algorithm, quickselect. median of medians

find the k-th smallest in an un-sorted array

median of two sorted array,k-th of two sorted array
http://leetcode.com/2011/01/find-k-th-smallest-element-in-union-of.html



Wednesday, October 29, 2014

Notes for Leetcode's blog

Sliding window(已看,已理解,未写)
http://leetcode.com/2011/01/sliding-window-maximum.html

方法1:暴力解
方法2:for 循环内每一步计算的都是上一步的window的最大值,所以最后一个window得单独计算
注意: priority_queue每次push或者pop都会重新生成最大/小值在开头
方法3:用双向链表,queue里存的是element的index。当前的最大值永远在最前,不用考虑比最大值小的值。每当插入一个新的值,pop掉所有比它小的值。之后在对比当前最大值是否在范围(window)内

Sunday, October 12, 2014

Notes on Notes -- Chapter 2, 3, 4*

chapter 2
设定dummy

----------

chapter 3
Tower of Hanoi
iterative solution
http://en.wikipedia.org/wiki/Tower_of_Hanoi
--------

chapter4

trie -- implementation
http://en.wikipedia.org/wiki/Trie

heap

priority queue

graph -- implementation
http://www.cs.bu.edu/teaching/c/graph/linked/



Dijkstra's algorithm

greedy

Check if a graph has a cycle
undirected: use DFS
directed: use Topological sorting

lowest common ancestor
http://leetcode.com/2011/07/lowest-common-ancestor-of-a-binary-tree-part-i.html

Find the immediate right neighbor of the given node, with parent links given, but without root node.( Ebay interview question)

Word Ladder II 

Saturday, June 28, 2014

Notes: Distributed System -- Chubby, Zookeeper

Use of Chubby or Zookeeper
  • Group membership and name services
    By having each node register an ephemeral znode for itself (and any roles it might be fulfilling), you can use ZooKeeper as a replacement for DNS within your cluster. Nodes that go down are automatically removed from the list, and your cluster always has an up-to-date directory of the active nodes.
  • Distributed mutexes and master election
    We discussed these potential uses for ZooKeeper above in connection with sequential nodes. These features can help you implement automatic fail-over within your cluster, coordinate concurrent access to resources, and make other decisions in your cluster safely.
  • Asynchronous message passing and event broadcasting
    Although other tools are better suited to message passing when throughput is the main concern, I’ve found ZooKeeper to be quite useful for building a simple pub/sub system when needed. In one case, a cluster needed a long sequence of actions to be performed in the hours after a node was added or removed in the cluster. On demand, the sequence of actions was loaded into ZooKeeper as a group of sequential nodes, forming a queue. The “master” node processed each action at the designated time and in the correct order. The process took several hours and there was a chance that the master node might crash or be decommissioned during that time. Because ZooKeeper recorded the progress on each action, another node could pick up where the master left off in the event of any problem.
  • Centralized configuration management
    Using ZooKeeper to store your configuration information has two main benefits. First, new nodes only need to be told how to connect to ZooKeeper and can then download all other configuration information and determine the role they should play in the cluster for themselves. Second, your application can subscribe to changes in the configuration, allowing you to tweak the configuration through a ZooKeeper client and modify the cluster’s behavior at run-time.
Reference:
http://blog.cloudera.com/blog/2013/02/how-to-use-apache-zookeeper-to-build-distributed-apps-and-why/

http://www.ibm.com/developerworks/library/bd-zookeeper/

awesome read:  https://developer.yahoo.com/hadoop/tutorial/module6.html#intro
-------------------------------------------
group membership:
example
The Group Membership Service (GMS) uses self-defined system membership. Processes can join or leave the distributed system at any time. The GMS communicates this information to every other member in the system, with certain consistency guarantees. Each member in the group participates in membership decisions, which ensures that either all members see a new member or no members see it.
The membership coordinator, a key component of the GMS, handles "join" and "leave" requests, and also handles members that are suspected of having left the system. The system automatically elects the oldest member of the distributed system to act as the coordinator, and it elects a new one if the member fails or is unreachable. The coordinator's basic purpose is to relay the current membership view to each member of the distributed system and to ensure the consistency of the view at all times.
Because the SQLFire distributed system is dynamic, you can add or remove members in a very short time period. This makes it easy to reconfigure the system to handle added demand (load).The GMS permits the distributed system to progress under conditions in which a statically-defined membership system could not. A static model defines members by host and identity, which makes it difficult to add or remove members in an active distributed 

------------------------------------
Note:
write: only through master
read: any nodes can handle it

use: leader election, lock, consensus/voting?
more uses: http://ksjeyabarani.blogspot.com/2012/11/zookeeper-as-distributed-consensus.html
zookeeper recipe: http://zookeeper.apache.org/doc/current/recipes.html

Saturday, May 10, 2014

Notes: Distributed System -- Paxos

what is instance
  • An instance is the algorithm for choosing one value.
  • A round refers to a proposer's single attempt of a Phase 1 + Phase 2. A node can have multiple rounds in an instance of Paxos. A round id is globaly unique per instance across all nodes. This is sometimes called proposal number.
  • A node may take on several roles; most notably Proposer and Acceptor. In my answers, I'll assume each node takes on both roles.

A naive implementation of Paxos is not guaranteed to terminate because multiple nodes can leap-frog Prepare phases. There are two ways of getting around this. One is to have a random backoff before starting new Prepare phases. The second is to route all requests to a designated leader, that acts as proposer (The leader is chosen by a Paxos instance. See also Multi-paxos) 

roundId = i*M + index[node]where i is the ith round this node is starting (that is i is unique per node per paxos instance, and is monotonically increasing).

reference:  http://stackoverflow.com/questions/5850487/questions-about-paxos-implementation/10151660#10151660
----------------------
more paxos: Michael Deardeuff on stackoverflow
pseudo code
http://stackoverflow.com/questions/14435646/paxos-value-choice/14472334#14472334
 --------------------
Round ids (aka proposal numbers) should be increasing and must be unique per instance across all nodes.

-------------------------------
Paxos notes in Chinese
http://blog.csdn.net/m_vptr/article/details/8014220
 -------------------------------------------
No timeout in Paxos???? see below
----------------------------------
finish How to Build a Highly Available System, from section 6.4

------------------------------------------------------
lessons-learned-from-paxos: http://blog.willportnoy.com/2012/06/lessons-learned-from-paxos.html
explained a more practical Paxos

You may even choose to introduce nondeterminism: for example, I use randomized exponential backoff as a response to timeouts to allow a set of Paxos replicas to settle on a persistent leader allowing for faster consensus.

To make code deterministic for repeatable tests, it must have the same inputs. And the same inputs means the same network i/o, thread scheduling, random number generation - effectively all interaction with the outside world.

about timeout in Paxos
  1. Many theoretical algorithms from distributed systems use the Asynchronous Model of time, where there is no shared notion of time across different nodes. One technique used to reduce these methods to practice is to introduce timeouts on certain operations. There would be a timeout for the replica trying to become leader, a timeout for the non-leader replicas watching for a leader replica that has gone offline, and a timeout for the phase 2 rounds of voting. There is an important detail in the last timeout though - the better implementations of Paxos allow multiple instances to reach consensus in parallel (without picking a fixed factor of concurrency, as Stoppable Paxosdescribes). But it simply takes longer, even if purely by network latency, to drive more instances to consensus. So in the end, you either vary your timeout in proportion to the number of requests or you limit the number of concurrent requests so that the timeout is sufficient.

#11 how to avoid massive amount of messages passing in phase1
#15 queue with timeout can work as batching

----------------------------------------




Sunday, May 4, 2014

Hashing

hash function
page 264, clrs
h.k/ D bm.kA mod 1/c

Division Method:
h(k) = k mod m
where m is ideally prime

Multiplication Method:
h(k) = [(a k) mod 2w] (w >> r)
where a is a random odd integer between 2^(w-1) and 2^w, k is given by w bits, and m = table
size = 2r.

universal hashing

----------------------------------
Karp Rabin_wiki, Rolling Hash_wiki

However, there are two problems with this. First, because there are so many different strings, to keep the hash values small we have to assign some strings the same number. This means that if the hash values match, the strings might not match; we have to verify that they do, which can take a long time for long substrings. Luckily, a good hash function promises us that on most reasonable inputs, this won't happen too often, which keeps the average search time within an acceptable range.

hash functions used for karp rabin, go to wiki hash function
The key to the Rabin–Karp algorithm's performance is the efficient computation of hash values of the successive substrings of the text. One popular and effective rolling hash function treats every substring as a number in some base, the base being usually a large prime. For example, if the substring is "hi" and the base is 101, the hash value would be 104 × 1011 + 105 × 1010 = 10609 (ASCII of 'h' is 104 and of 'i' is 105)

multiple pattern search
The Rabin–Karp algorithm is inferior for single pattern searching to Knuth–Morris–Pratt algorithm, Boyer–Moore string search algorithm and other faster single pattern string searching algorithms because of its slow worst case behavior. However, it is an algorithm of choice for multiple pattern search.

------------------------------------------
Probing 
Liner probing
quadratic probing 
Double hashing
h(k, i) = (h1(k) + i * h2(k) mod m